Josh,
You can use any soft metal hammer or mallet with your brass punch.
Consider brass, copper, aluminum, lead... even hard rubber or plastic mallets too. There are also dead blow hard rubber mallets with lead shot in them.
Any of these would be fine with the brass punch.