In 1912-1913, the state purchased land from A.T. McKinney to expand Sam Houston Normal Institute southwards. In 1913, the three-story Women’s Gymnasium was built on the new land, becoming the fifth permanent building on campus.

The first floor of the Gymnasium housed one of the first indoor swimming pools in East Texas, though was later covered with a gym floor. The second floor contained classrooms, offices, and the gym, while the third floor contained the gym gallery and a large classroom.

In 1960, an addition to the Women's Gymnasium extended the building northward toward the Peabody Memorial Library. The University Post Office was additionally located in the basement of the gymnasium at one point in time.

In 1983, the Women's Gymnasium was demolished to make room for the Dan Rather Communications Building as part of new Campus Master Plan. Classes from the men's and the women's gymnasiums were relocated to the Health-Kinesiology Center upon its completion in 1985.

The Men's Gymnasium was scheduled for demolition after the completion of the Health-Kinesiology Center. Instead, the building (renamed Academic Building Three/AB3 in October 1987) was used for the Office of Graduate Studies and the McNair Scholars Program, ROTC in 1986, dance, and afterward for the Department of Nursing until the construction of the Nursing Biological Sciences Building in 2010.

The cornerstone of the Women's Gymnasium, along with the Education Building's cornerstone, is on display in downtown Huntsville as part of the Sculptural Wall outdoor project by Richard Haas.