I actually helped build a hunting camp a few years back. It was somewhat close to the road, but the closest power pole was 3/4 of a mile away, and the power company wanted $5k+ to run new line. We hooked just about everything up to propane. Small RV propane fridges are cheap and readily available. We also put in propane lights and a stove. Solar lights are so cheap now, I'd probably get a couple of the ones with the detachable panels and use them. We wired some stuff up to run a generator if needed, but I don't think we've ever used it.
Another camp I stayed at down the road, the guy built a small barn/shed to store his ATV in, and sectioned off a portion of it for a generator. He extended the muffler out the back, facing away from the camp, and added a little flip-up door for fresh air in the front. UF wire to the camp. It was a $300 3500 watt Champion from Lowes that can run 14 hours on 4 gallons of gas. For another $100 he bought a tri-fuel kit so it could also run on propane if needed. We'd shut the generator down when going to bed, but it wasn't very loud to begin with.