No family in colonial Virginia was more prominent or more powerful than the Randolphs. "You must be prepared to hear the name Randolph frequently," wrote a French traveler to Virginia in the 1780s, the marquis de Chastellux, who recognized the Randolphs as "one of the most numerous and wealthiest" of the "first families" of the colony. When writing Moby Dick in 1851, Herman Melville cited the Randolphs as the quintessential "old established family in the land," the ultimate contrast to those families whose sons were forced into the perilous profession of whaling.
The dynasty was founded by William Randolph I, who in 1680 established a seat below the falls of the James River. Generations of his heirs developed neighboring plantations, eleven major ones in all. As owners of tens of thousands of acres of land worked by hundreds of slaves, and serving as well as merchants and shipowners, the Randolphs—unified like a modern business conglomerate—affected the economic affairs of the colony. They also influenced the administration of government. Among the Randolph politicians were some of the most accomplished lawyers in the region.
Art patronage by the family increased with its third generation, because of both inherited wealth and the availability in the 1750s of a highly talented portraitist. Randolph brothers and sisters awarded the artist John Wollaston multiple commissions. William Randolph III of Wilton ordered ten paintings. The inspiration for the Wilton collection and for the house itself was drawn from the spectacular gallery and mansion at nearby Westover.