James Drury was born on 18 April 1934 in New York City, where his father James, Sr., was a New York University professor. He grew up in both New York and Oregon. Drury contracted polio at the age of ten.
In 1959, Drury was cast as Harding, Jr., in the episode "Murder at the Mansion" of the David Janssen CBS crime drama, Richard Diamond, Private Detective. John Hoyt played Harding, Sr., with Frank Ferguson as Capt. Hoyt.
On May 9, 1959, early in his career, Drury appeared as Neal Adams in the episode "Client Neal Adams" of ABC's western series Black Saddle. In the story line, Adams is an old friend of series protagonist Clay Culhane, a gunfighter-turned-lawyer played by Peter Breck. Adams has robbed a bank of $8,000 and was subsequently shot in the back by a pursuing bounty hunter, played by Charles Aidman. Adams asks Culhane for help and makes the false claim that the bounty hunter is the brother of a man whom Adams had earlier killed in self-defense. From the start, Marshal Gib Scott, played by Russell Johnson, doubts Adams' story and questions Culhane's judgment in the matter.
In 1960, Drury appeared in different roles in two episodes, "Fair Game" and "Vindication" of another ABC western series, The Rebel, starring Nick Adams as a Confederate adventurer roaming through the post-Civil War American West.
On November 16, 1960, Drury delivered a solid performance as young pioneer Justin Claiborne in the episode "The Bleymier Story" of NBC's Wagon Train, broadcast just days after the death of series lead Ward Bond. Robert Horton as the scout Flint McCullough struggles to get a wagon train through a Sioux burial ground. Dan Duryea played the mentally unstable Samuel Bleymier who is obsessed by demons and superstitions. Bleymier also opposes the interest shown by Claiborne to his daughter, Belle, portrayed by Elen Willard. The episode is filmed mostly during heavy rains, high winds, and a cyclone.
Drury played in secondary roles for Walt Disney. In 1962, he was cast in a substantial role as a lascivious gold prospector in the early Sam Peckinpah western Ride the High Country. In the same year, Drury landed the top-billed leading role of the ranch foreman on The Virginian, a lavish series which ran for nine seasons. The show was based on Owen Wister's classic novel, and the various screen versions that had been filmed since. It has been rumored that Drury was possibly cast because of a vague resemblance to Gary Cooper, who had played the part in an early film version. Director Andrew J. Fenady stated that Drury was offered the lead role in the series because of his impressive performance in an unsuccessful television pilot The Yank that was inspired by Nick Adams' The Rebel, on which Drury had guest starred twice. In the series, as in the novel, the actual name of "The Virginian" is never revealed. Drury's costars on The Virginian included Lee J. Cobb, Doug McClure, Randy Boone, Roberta Shore, Gary Clarke, Clu Gulager, Diane Roter and toward the end of the run, Tim Matheson.
The Virginian was one of the few 90-minute series on television in the 1960s. It running time, plus high number of episodes per season (up to 30), created challenges with the shooting schedule. In a 2013 interview, James Drury recalled:
"The majority of...episodes were shot in eight days. When we started out, they took about ten days. They decided they couldn't afford that much time because we aired the episodes every five days (excluding weekends). To keep up with the airing schedule, we had to run multiple units--as many as four or five different episodes filming at the same time. I would ride my horse or take the studio limousine back and forth between the sets to do my two line piece in one episode, ten pages of dialogue in another episode, do a cattle drive in another episode, a wild horse drive in another, and then a gunfight and a robbery in yet another episode. I had to keep everything straight and it was absolutely no problem and a joy to do. I would do it all over again tomorrow."
In 1993, Drury had a guest-starring role as Captain Tom Price on the first three episodes of Walker, Texas Ranger, opposite Chuck Norris and Clarence Gilyard. Drury also had a cameo role in the 2000 TV movie of The Virginian starring Bill Pullman. The film followed Wister's novel more closely than had the television series. Drury appeared in a number of films and other television programs, including The Young Warriors and the TV cowboy reunion movie The Gambler Returns: The Luck of the Draw with Doug McClure, who played the character Trampas during The Virginian.
In 1991, Drury was inducted into the Hall of Great Western Performers at the National Cowboy and Western Heritage Museum in Oklahoma City. In 1997 and 2003, he was a guest at the Western Film Fair in Charlotte, North Carolina. At the 2003 show, he was reunited with his Virginian co-stars Gary Clarke, Randy Boone, and Roberta Shore.
Drury was in the oil and natural gas business in Houston at the turn of the 21st century. His son, Timothy Drury, is a keyboardist, guitarist and vocalist who has played with the rock groups The Eagles and Whitesnake. Drury was a close friend of the Houston marksman Joe Bowman, who trained numerous actors on how to use weapons. Drury is also the longtime best friend of both Robert Horton and Robert Fuller, since 1954, where the three met while under contract at MGM Studios. Together (along with Horton, until his final appearance in 2011), they attend the annual Festival of The West in Scottsdale, Arizona, among other Western Festivals. He's also the longtime friend of James Best, an actor and acting coach, who is eight years Drury's senior. He and Best met in 1965 on the set of The Virginian.