James Gilliam Gee was born in Union County, South Carolina, on August 20, 1896, to James Monroe Gee, a judge, and Elizabeth Farrar. James' nicknames during his life were Jimmy and Mutt. James had one sibling, Clough Farrar Gee, who was a major in the U.S. Army.

James and his brother Clough attend Clemson Agricultural and Mechanical College, now known as Clemson University. James won a four-year scholarship to Clemson by taking a competitive examination. During James' time at Clemson, he played football as a fullback, center, and guard from 1914 to 1917 and played baseball as a right fielder in 1917. James was awarded the College Football All-Southern Team in 1917.

Clough graduated in 1915, and James graduated with a bachelor's in chemistry and was third in his class in April 1917 before the United States entered World War I. In his early twenties, James entered the war effort of the First World War. He was assigned to the 2nd South Carolina Infantry and became a battery commander, attaining the rank of major.

In 1919, James received a graduate fellowship to attend Cornell University and earned his master's in agricultural economics and vocational education. After James graduated from Cornell University, he played professional baseball for Binghampton, New York as a catcher and an outfielder in the spring of 1920. Playing professional ball for a short time, he took a faculty position at Sam Houston State Teachers College teaching agriculture education. Along with teaching, James was the college's head football coach from 1920 to 1922.

On December 14, 1922, James married Cecile Gibbs, who was from a prominent family in Huntsville, Texas. Cecile's parents were Thomas Clifton Gibbs and Jamesetta Hunt Adams. James and Cecile had one son, Thomas Gibbs Gee, born on December 9, 1925.

From 1925 to 1926, Gee was the line football coach at the University of Florida and Head of the Department of Vocational Teacher Training Agriculture. From 1927-1930 James was the Athletic Director and Head of the Department of Physical Education at Clemson College.

After his time at Clemson, James attended Harvard University for graduate work, then George Peabody College for Teachers (now known as Vanderbilt University) to earn his Ph.D. in educational guidance. After receiving his Ph.D. in 1933, James returned to SHSTC to teach vocational and educational guidance. In 1937, he was promoted to Dean of the College.

Once again, James entered the war effort when the United States became a part of the Second World War. He rose to the rank of colonel in the infantry, and while serving under the famous General George Smith Patton Jr., he earned four combat stars during the European Tour.

After World War II, James returned to SHSTC and became acting president during President Charles Shaver's tenure for a brief time.

In September 1947, James became the fifth president of East Texas State Teachers College. During his tenure, the college became desegregated, the name changed to Texas A&M Univesity-Commerce, and the campus expanded through construction of new buildings. Gee held this position until his retirement in 1967.

On May 4, 1967, James was awarded an honorary Doctor of Law from his alma mater, Clemson Agricultural and Mechanical College (Clemson University).

In 1975, James entered the Clemson Athletics Hall of Fame.

James Gilliam Gee, 86, died on October 29, 1982, and his wife, Cecile Gibbs Gee, 95, died on June 13, 1994. They are both buried in Oakwood Cemetery, Huntsville, Texas. Throughout James' life, he was a member of the American Legion, Reserve Officers Association of the United States, Alpha Gamma Rho, Kappa Delta Pi, Phi Delta Kappa, Texas State Teachers Association, Kiwanis Club, and the National Education Association.