The house was built in 1893 for George and Sarah Morgan, sister of J. P. Morgan, to designs by architects Rotch & Tilden. Its exterior is brick with brownstone trim, containing approximately 50 rooms in a total of 28,000 square feet (2,600 m2) of living space, including 9 main bedrooms and 10 servant’s bedrooms, 7 bathrooms, and 17 fireplaces. The house was set within a large landscaped garden of 26 acres (since reduced to 11.7 acres).
A smaller home was moved off of the property and across the street prior to the construction of Ventfort Hall. This home was owned by the Haggerty family and known as Vent Fort. The colonel of the 54th Massachusetts Volunteer Infantry and civil war hero, Robert Gould Shaw, spent his brief honeymoon here with his wife, Annie Kneeland Haggerty. After Robert’s death, Annie moved to Europe. She did not return to Vent Fort until her last two summers, when she rented the memory-filled home from the Morgan family.
After the Morgans’ deaths, the house was rented for several years to Margaret Vanderbilt, then purchased in 1925 by W. Roscoe and Mary Minturn Bonsal who in turn sold the house in 1945, after which it served as a dormitory for Tanglewood music students, a summer hotel, the Michel Fokine Ballet Summer Camp, and community housing for the religious organization The Bible Speaks (now known as Greater Grace World Outreach).
In 1991, a nursing home developer planned to demolish the building. Over the following years, the house sustained significant damage, the paneling was stripped from the walls, and part of the roof collapsed. In June 1997 it was rescued by the Ventfort Hall Association. The non-profit has been repairing the damage, as well as trying to establish a national museum of the Gilded Age within its walls.