It all sounds lovely and romantic, and you may think you have what it takes, but...
I've done this twice now, on a 5 acre property (which 2nd wife & I now live in) and prior to that a 10 acre property (that 1st wife and I owned). (Note clue right there.) I'm an engineer and spent a career in building construction so had a decent idea of some of the things I'd be getting into, or so I thought. So have a few comments to pass along. Apologies for being long-winded.
The 5 acres was bare land with no trees because it had been farmed before, plus it had a small crappy old farm house that eventually got torn down but was live-able until the new house was at occupancy permit stage. The 10 acres was heavily forested and we bought a well-used fifth wheel trailer to live in. Had it logged and we sold the timber at a good price which helped, but had a HUGE mess of stumps and brush to burn. Logging is not fun and best kept to a minimum in the beginning. Having no trees in the way was definitely a big help.
There will be all sorts of twists and turns and unexpected things that you could never have expected in your wildest dreams (or worst nightmares) and you should be prepared to be able to handle the worst. What do you know about soil conditions and water availability for a well? Will you have poor perc rates and have to install an expensive sewage treatment plant? Will a simple surface well work or will you need to drill in one or more locations for a deep well? What are neighboring properties like for wells? Drilling wells is expensive and sometimes you need to drill in multiple locations. Sometimes you can't get a building permit until you have a well and the water tests out okay to gov't standards and/or have a permit for the septic system.
Consider making an offer contingent on finding a suitable location for a septic field with a decent perc rate and also for finding a source of potable water. If the owner balks, maybe it's not the place for you. Some owner's provide an approved perc test and some have a well in place.
For the 10 acre lot, we had a standard builder's type mortgage and had to meet certain completion stages as we went along to get another draw and it was tough to get to each stage. On the 5 acre lot we had a line of credit type mortgage. The LOC mortgage worked really well. As the construction progressed, the property value increased and were able to get the limit increased when the bank's appraiser looked at it. When you are building on acreage, there are many things that add little, if any, value to the property. You can lay down tons & tons of expensive gravel for a driveway for ex. and not increase the value. Similar with fencing, logging, well, etc. There will always be unexpected expenses and if you had a conventional loan and had to meet specific completion stages, you could be SOL. Banks generally don't like owner-built projects, esp. if on bare acreage. In Canada, I think LOC mortgages aren't available anymore.
Do NOT build a barn, workshop, garage or outbuilding first and do NOT live in one of these temporarily. BTDT and it is a bad idea. Build the house first and get it live-able. Banks want houses, not outbuildings and some jurisdictions will not allow anything other than a house to be built first. Have a functioning septic system, a potable water supply (well or city water) and power for temp. accomodation. Live in an RV if you have to in the very beginning but you will need to dump the holding tanks every few days to a week. Getting a pump-out service is expensive. Maintaining a healthy relationship with g/f and family is important and should be at the top of your list and living in third world conditions is not helpful.
Unless you will be building a small house, consider paying a framing crew to get the structure up asap so you can get the roof on. Then you can take a deep breath and relax and take a break or go slower to keep your relationship on track. Unless you are a professional framer and have a bodybuilder's body, framing will drag out longer than expected. I framed the 4K+ sq. ft. house plus 2K sq. ft. separate workshop on the 10 acres and everything else except drywall and plumbing entirely myself and it was slow and painful. On the 5 acres, the house is 4K+ sq. ft. with 3K+ sq. ft. attached workshop/garage a framing crew built it and a roofing co. did the roof (metal) and once the roof was on, things went much faster and smoother. Do you know what soils condition is like for where the foundations will be? Is there blue clay or solid rock below the topsoil? Both of these can cost a lot to remedy. Building a house closer to the street can sometimes save a lot (driveway, logging, temp. & permanent power).
If you are going to be buried in the project outside your normal job working hours, maybe set your g/f up as the official purchaser and gofer. That way you can find or negotiate better deals & prices on materials and not be without things like nails that can stop your work. I used to work all day on the house and then run to HD late in the evening to get misc. stuff but paid higher prices for it. Set up accounts with a lumber yard, plumbing & electric wholesalers, etc. in advance. Cash accounts can get you a decent discount sometimes without having to be in debt to them. My 2nd wife is an accountant which has helped tremendously - wouldn't let me buy anything without shopping around for the best price. She even found a few things used off CL for cheap like perfectly good fiberglass batts from a house being torn down. Bought some things from a building materials salvage place like all the 3" rigid foam under the garage floor. Every dollar that you can save along the way can make a big difference in the end.
The inspection & permitting processes by the various local AHJs will take a long time, longer than you think - covering the septic system, well, logging, buildings, importing fill, etc. and you will be hog-tied until you wait out the process and have permit in hand. The AHJ *could* require a structural engineer, a geo-tech. eng. for soils compaction or drainage, a surveyor, etc. all costing more $$. Choose contractors wisely and avoid issues or at the worst, a lawsuit. Be prepared for some things costing more, and maybe some a lot more than expected. Think about having some sort of contingency fund available.
Don't know if you are in an agricultural land area. We grow hedging trees and have an area in hay to get farm tax status. Doesn't make much money and is plenty of work but helps a LOT on reducing the property tax. After 13 years of working on our current place, we just had it appraised and it's now worth almost 10 times what it cost in 2003. The ultra-red hot local housing market in the last few years has really helped but if we were to be looking for the same kind of property to build on today from scratch, I don't think we could afford it. Now have a good retirement fund for when we eventually sell and downsize.
Executive summary: Focus first on maintaining healthy relationships with g/f, family, friends and those you work with. Secondly,, focus on getting the house up, and only things that must be done to support it like well, septic, power and driveway and do not let yourself get distracted by other things that will slow down the house and **** up your time and available $$.
Just my 2 cent opinions...
From what I've read, "barndominums" are a new fad - that means no comparables, which means likely low appraisal on the end product which limits your loan value - possibly not even to cover construction costs.