The original castle, probably wooden, is first mentioned in 1260 when it was besieged and burned to the ground, after which it was rebuilt in stone. In 1280 this second castle was again besieged and this time the bailey was burned down. This castle probably consisted of a simple hall-keep, two stories high with 1.20 meter thick walls, and featured a surrounding moat which was fed by the nearby river Rhine.
During the 14th century the castle was continually enlarged. Doorwerth Castle was originally the property of the Van Dorenweerd family. In 1402 Robert van Dorenweerd dedicated the castle to the Count of Gelre, Reinald IV. In return Robert was granted the castle and its land in fief. Around the middle of the 15th century the castle was enlarged again, this time by knight Reinald van Homoet, the 10th Lord of Dorenweerd, who was also the owner of Doornenburg Castle.
Doorwerth Castle reached its largest form just after the middle of the 16th century under Daem Schellart van Obbendorf, the 15th Lord of Dorenweerd. He made the castle and the group of buildings on the bailey into a unity and adjusted them for more space and comfort. By 1560 Doorwerth Castle had almost reached its present appearance. Around 1637 the bailey was rebuilt to its present appearance and a dike was built around the castle to protect it from flooding of the river Rhine.
Shortly after, the castle changed ownership due to financial problems and was granted in fief to a German Count, Anton I van Aldenburg. His successors did not alter the castle or bailey but did acquire more land. At the end of the 18th century the castle was no longer inhabited, but was looked after by a steward for its owners who now lived in England.