Some folks are impossible to please. You may fall into that category.
You ran off your $400 quote. Don't expect it to get any better than that.
The fee is for more than just his time on site. You're paying for tools, risk, the time he spent talking to you about the job, insurance, and the...