A regular ball pein hammer is the tool you need!
The face of a ball pein hammer will be harder than the haft of a punch or chisel, but not as hard as a carpenters hammer, so there is no risk of chips. Also, the face will be flat, or even slightly concave, whereas some carpenters hammmers have slightly convex faces!
I use 8oz and 16oz the most. Probably use one or the other for 90% of jobs. I tend to use the 8oz more for general metalwork (centre punching etc) or for punching small pins. The 16oz wins for automotive. I also have 12oz, 2lb and 3lb.
Being a Brit, most of mine are British brands, though I have a couple of Vaughan hammers and they are superb. If I was in the U.S. I wouldn’t look further than Vaughan or Estwing. Both superb makers, and reasonable prices.
Worth thinking about the handle material. For use at base, I generally prefer a traditional wooden handle. They don’t come loose if you get one from a decent maker, the balance is good, and they absorb some shock. If you travel with tools in a box though, they need to be well protected, so one of the one piece steel designs might be a better choice. I prefer glass fibre or composite though, as they absorb the shock better.
For heavy punching, a dead blow hammer with a hard face is the newest concept, and I have to say these are the most comfortable option to use. If you’re doing a lot, probably the only option. I probably wouldn’t use one for centre punching etc though!