Inchcolm lies in the Firth of Forth off the south coast of Fife opposite Braefoot Bay, east of the Forth Bridge, south of Aberdour, Fife, and north of the City of Edinburgh. It is separated from the Fife mainland by a stretch of water known as Mortimer's Deep. The island forms part of the parish of Aberdour, and lies a quarter of a mile from the shore. In the days when people were compelled to cross the Firth of Forth by boat as opposed to bridge, the island was a great deal less isolated, and on the ferry routes between Leith/Lothian and Fife.
The island can be broadly divided into three sections: the east, where its military defensive operations were centered during the Second World War, the lower central part, with the small natural harbour and shop, and a larger western end.
Between Aberdour and Inchcolm is the channel called "Mortimer's Deep". It is believed that this was named after William de Mortimer