My detached shop was built when my house was built. This builder doesn't normally finish the detached garages and I was planning to do it myself, until I got the builder's quote: $1200 bucks. The was for a 24'x24'x12.5' shop; they insulated the walls and attic, mud and finished the Sheetrock, painted, and even installed base molding.
This was 7 years ago, and maybe they gave me a break on price or covered it somewhere else in the, either way I could not have done the job for that price.
I think $4k, even in Long Island, is way too much. As someone mentioned they might have quoted the high price to see if you'd bite. We have a ton of construction work available here in northern Alabama and I know some contractors pick and chose their work. Call a contractor around here that advertises "no job too big or too small" with a small job and after they stop by for a quote (if they even show up), you'll never see them again.