Well, that sounds like a lot of back fill. And then it has to be tamped and allowed to settle. There are different ways of doing this where you build it up and tamp it every four inches or so, and different types of fill.
Or you go with a full foundation. Then you fill that. Excavators do this all the time. My excavator suggested letting it settle over the winter, and building in the spring. If you do it right, you don't have to wait. That means correct fill and correct tamping.
I built my site up about five feet over a 30 foot span. Put a floating slab over that and stick built on that. It hasn't settled in 14 years. It cost me as much to fill and pour the slab as it did to build the structure. More in fact.
In the end it would have been just as easy to go with a full foundation, but this way I get to walk all around the building, and I have a back equipment shed that leads out to the side of the property. I was happy with the floating slab, as it has a huge footprint to spread the load. My excavator was not familiar with this type of foundation back then, so I had to spec everything and sign off on it.
I was comfortable doing that, but this is in fact why they have engineers. In a situation like yours, if I wanted to back fill and build a pole barn, I would probably back fill to a known depth and four two long footings to carry the poles on each side of the barn. Then I would back fill the rest and dig out the holes to the footings later. I do stuff like that, and have in fact done that in the past. Because I am not an engineer, I tend to overbuild things that no one can question later on.
You, OTOH, should have an engineer, or at least the advice of an excavator and foundation specialist. The leaning tower of Pizza was in fact already leaning before they even finished building it. You don't want that.