The Pepergasthuis was founded in 1405 by Berneer Solleder and his son Albert. Originally it served as a guest house for pilgrims who came to Groningen, because the Martinikerk kept a relic of John the Baptist that attracted many pilgrims to Groningen. Because of that destination the the guest house was named after Gertrude of Nivelles, the patron of the travellers. But the guest house had financial problems, so people like Pope Sixtus IV donated money. Menso Jetlema donated estates, but wanted the guest house to give a ton of herring to the poor people of the village of Faan every year.
After the capitulation of Groningen (the Spanish Occupier capitulated for the army of Maurice of Nassau and William Louis, Count of Nassau-Dillenburg) in 1594 the complex, like all Catholic buildings, got a new destination. The guest house changed into a residential complex for older people of the city. People of 50 years and older could buy a spot in the guest house. They didn't only get a place to live till their death, but also care. The residents were called conventualen.
Next to that a part of the complex was set up as a dolhuis, a house for mentally ill people. The people who were housed here were exhibited for pay on Sundays. In 1702 they were moved to a new guest house at the Rademarkt.
The complex has been extended several times, most recently in 1861.