Not allowing an accessory building to be more forward than the front line of the primary structure is standard boilerplate almost everywhere, not something they decided to do in your area because of seeing too many barns. That's a funny comment anyway, as the ag zoning is for properties to have barns.
I am curious why you still seem intent on a detached building. I understand that you would think that with a 2 acre lot, you should have lots of leeway on where to put an accessory building. But the combination of the location of the existing residence, so far into the back corner of the lot, and the new larger setback lines, takes away a lot of options.
You really have 2 options.
1. locate the detached building where you really want it, and spend the time, money and effort to try for a variance.
2. Attach the new structure to the existing one, on either side, allowing you to move the front face of that combined structure as far forward (Toward the private drive), as needed, up to a line 50' from the private drive easement or right of way.
To a designer with lots of experience in these kinds of situations, I would pick the attached option.
Most places, a variance is seldom given. You have to prove that there is no other alternative, and you have to show a hardship (Not financial). Basically you need to show that you are prevented from using your property in the ways that the zoning allows by special circumstances.
I have gotten variances that were very difficult and demanding. It involved careful planning, good politicking with neighbors, excellent presentation, and the threat to the city of a lawsuit for "Taking without compensation".
I've also seen large sums of money, time and effort spent, to no avail, because the city's attitude was that "We don't give variances". Knowing your city and neighbors is critical.
You can spend the money and spin the wheel. The attached option is always there as a backup plan.
Bill