Quick get it done before anyone notices.
Since you made such a big point about wanting to avoid all the necessary red tape to tear down and start over as compared to a re-model, I hope you decided to get all the necessary permits, as in essence, you decided to do a tear down. Even if you are making the case that it's a "remodel", (given the small amount of the building still standing,) you are now dealing with a giant "addition". Around here, even the smallest addition means permits and inspections all along the way. Doesn't your town require this too???
In my town, it's been know to happen that the building inspector has required finished, and very expensive, (and unpermitted), additions to be completely torn down. Brand new unpermited bathrooms have been completely torn out. There
is some sort of "post-completion" permiting process, but that is usually reserved for folks who unwittingly buy a property that had unpermitted work...and assuming that work is fully compliant anyway.
If you're taking the risk that no one will notice...best of luck. If you reuse all that old lumber on the exterior, that could help your chances. But it still seems like a massive risk.