Masjid Zahir was built in 1912 on the initiative of His Highness Tunku Mahmud Ibni Almarhum Sultan Ahmad Tajuddin Mukarram Shah. The site of the masjid was previously the resting place of Kedah's fallen worriors during the course of defending Kedah against Siamese invasion (1821).
The architectural design of the masjid was taken from Masjid Azizi in Bandar Langkat, Southern Sumatera. The masjis is decorated with five main domes representing the 5 islamic precepts and it was officially open on Friday, 15 October 1915 by Almarhum Sultan Abdul Hamid Halim Shah. His Royal Highness was also the Imam for Friday prayers on that day after the reading of the sermon by His Highness Tunku Mahmud.