An isolated mansion on the south side of the harbour was once reputed to be used for smuggling. It belonged to the owner of Fast Castle, Robert Logan of Restalrig (d 1606), who was involved in the Gowrie conspiracy (a bloody episode in Scottish history). Seven years after his death, his body was exhumed, judged and found guilty.