I have thought about the same thing, but for budgeting reasons. I am thinking about buying the land, and putting up a garage with an apartment to live in (I'm single) upstairs. Build the house later, then rent out the apartment as income.
More to your question, I know two people who lived at...