Bucket mud is water-based. It hardens as the water evaporates, meaning that it will shrink somewhat as it dries. This is great for paper Tape as it “*****” the tape in to the seams. However, as you can imagine, this makes it a poor choice for filling in holes and other larger imperfections.
Setting mud, aka hot mud, ezsand/durabond, etc, is like Portland cement in that it hardens via hydration—the water is incorporated into the final product. You won’t experience anywhere near the degree of shrinkage but it’ll still take two coats.
Spackling compounds are fine for small, clean holes where you can just fill the hole in and be fine. Like 1/8” or so. it’s nice to crack open the container, fill a couple small holes and put it away until next time. it isn’t strong enough for the larger holes you describe though.
If you want to do it right, fill the bigger holes with durabond. It’s strong as hell, originally designed to be applied between vertical sheets of drywall so that they wouldn’t sag over time. It sands about as well as concrete though, so apply it a little shallow. Follow up with 1-2 coats of hot mud. You’re gonna have to paint anyways and you’ll likely notice a bunch of extra imperfections once you get to work, so might as well mix up a pan of mud, put the mariachi music on, and get cracking