The plan of Virginia House was based on the Weddell's wish that someday their home would be the headquarters of the Virginia Historical Society. The large library on the second floor and the archival storage in the basement were constructed with this idea in mind. The Gallery was originally intended for use as an exhibit space for the Historical Society collections. It may be that the Weddells chose Henry Grant Morse as their architect because of Morse's experience designing public buildings. Morse had drafted the plans for the Carnegie Public Library in Norfolk, Virginia, and the Carnegie Free Library in Camden, N.J.