The caves are near the gorge bottom on the south side. Cox's Cave was discovered in 1837 and Gough's Cave in 1890. The series of chambers follow the course of underground streams through the porous limestone. The stalagmites, stalactites, petrified falls, lace curtains and pillars are coloured according to the minerals in the limestone: rust red (iron), green (manganese) and grey (lead).