While I see you point, there is a limit to what can be brought up to code with in reason and most inspectors here seem to understand that and are willing to make some exceptions. Furring the 2x3s up to a 2x4 will do nothing for improving the structure and in the 80+ years it's been standing on 2x3s there's no evidence of the structure being inadequate. My main reason for wanting to add the 2x4 wall is to gain extra R value and allow me to make the walls nice and true for finishing. Gaining structural strength is a welcome bonus and I doubt the 2x3s are going to start being an issue anytime soon.
I wasn’t trying to start a big argument about this, it’s just that I know from personal experience that around here when you remodel/renovate something you need to do it in accordance with the current building codes. I will give you a perfect example of this.
A good friend of mine has an older home and his wife has been on his case for a long time about remodeling their kitchen and getting all new appliances. He finally decided to do the remodel his wife has been nagging him about and he asked me to help him. I told him I would help as much as I could.
He went down and talked with the city building people and got the permits he needed and then he went with his wife and picked out all the stuff they wanted for the new kitchen.
We ripped down all the walls, (lathe and plaster) which was a real dirty/messy job and then threw it all out into a big dumpster he had procured from the waste disposal people. The wiring he had previously was only 2 wires but he did have 100 amp service and there were a few extra circuit breakers in the electrical box. Since we were adding some new circuits, he had to have an electrical permit and he had to bring the whole kitchen up to code and have it inspected.
The studs in the walls were not all the same, some studs were 12 inches on center, others were 24 inches on center, and the rest were all sorts of different distances apart and the inspector said we could have extra studs, but we had to at least have a stud every 16 inches. One problem we had was the studs were actually 2X4 so we had to use 2X6’s and cut them down with a table saw to be the right size, depth wise. Anyway, we ended up removing the old studs one at a time and putting in all new ones so that we could insulate with regular rolls of fiberglass insulation. Took us awhile to do that but it worked out really well.
Since we were working with the plumbing as well, we had to follow all the current plumbing codes for the whole kitchen. We ran plumbing for an ice maker/water in the door refrigerator and we replaced all the water lines in the kitchen and drain lines as well. We also ran power and water for the new dishwasher (they didn't had one before) and we even put in a garbage disposal, which I am not big fan of. Everything had to meet the current codes for plumbing.
In short, everything we worked on in the kitchen had to be brought up to the current building code and everything had to be inspected to be sure we did the work correctly.
That’s why I said what I did, around here if you remodel/renovate a building, whatever parts you are working on, when you are finished, it has to meet the current building codes. Things may be different where you live, I don’t know. All I can tell you is how it is around here but I think most places would have similar rules.