The principal residence of the Earls of Morton since 1642, the castle was abandoned in favour of Aberdour House in 1725. The present castle has a primitive, 14C tower, the upper part of which was rebuilt in the 15C. The land south of the castle was laid out in terraces, behind which were a dovecote and an orchard. The 16C terraced gardens have been restored.