The house at Coleton Fishacre was built as a country home for Rupert D'Oyly Carte and his wife, Lady Dorothy Carte, between 1923 and 1926. The architect was Oswald Milne, a former assistant to Edwin Lutyens, who designed the house with the principles of the Arts and Crafts Movement in mind: simplicity of design and quality of craftsmanship. The influence of this older movement notwithstanding, the house is influenced by its own time, especially in its Art Deco interior. The structure is built of local slate rubble with a Delabole slate roof.
Although built as a country home, Lady Dorothy lived in the house as her primary residence by the later 1920s. After the Cartes' divorce in 1941, their daughter, Bridget D'Oyly Carte, took over the house, which her father, who lived in London, would visit for long weekends. She sold the house in 1949 after his death.
The garden at Coleton Fishacre runs down a narrow combe from the house to the sea at Pudcombe Cove. The garden, originally planted by Lady Dorothy, features rare and exotic plants, some of which are unusual in their ability to grow outside a tropical climate due to the proximity of the Gulf Stream to this part of the coast of Devon. Lady Dorothy was noted for retrieving exotic plant species for the garden during her journeys abroad. The Cartes employed a staff of six to maintain the garden, compared with a staff of four to run the house.