The owner of the old castle, Walter Raleigh, had a new residence built in 1594, Sherborne Lodge, forming the nucleus of the current building on the far banks of the Yeo. Sir John Digby extended the castle in the 17C, remaining faithful to its original style. The residence, which has stayed in the Digby family ever since, houses fine collections of paintings, furniture and porcelain. The grounds were laid out by Capability Brown in 1776-1779.