Andrew Female College, located in Huntsville, Texas, was founded in 1852 and charted by the Texas Conference of Methodist Churches on February 7, 1853. The college was named after James Osgood Andrew, a Methodist bishop who had caused a split in the Methodist Episcopal Church due to his ownership of African American slaves. This split helped to build a new Southern branch of this denomination.

In May 1853, Andrew Female College held its first five-month session in what is known as Huntsville's old "Brick Academy." Enrollment at the college was high, and quarters quickly became overcrowded, but due to the residents of Huntsville, funds were raised to build a two-story building which was completed in 1855. In the session of 1856 - 1857, eighty female students enrolled in the college from Walker County and surrounding counties. The college's first president was James M. Follansbee.

The curriculum for the students focused on classical literature, languages, art, music, and domestic life. The college also hosted annual concerts and exhibitions sponsored by the Music and Art departments. During these events, people close by and far away would come to witness the talents of the college's students.

The college was able to stay open during the Civil War due to the efforts of the school's board of trustees Charles Keenan, Daniel Baker, Henderson Yoakum, and Thomas Ball. Although the college was well off during the years of the Civil War, it was the 1867 yellow fever epidemic that helped lead to the school's closure in 1880. During the epidemic, yellow fever claimed the lives of the college president, several members of the faculty, and several of the students. Even though the college stayed open for twelve years after the epidemic, the college was never quite the same. In 1879, Andrew Female College had to compete with the opening of Sam Houston Normal Institute, which finally caused the college to close in 1880.

After the closure of Andrew Female College, the property was conveyed to the City of Huntsville and reopened later that year as the city's first public school. The structure was later relocated and became a public school for African American children.