One thing you want to watch out for is the broach going crooked in your arbor press. Just make sure you don't horse it down through there all at once. Myself, I like to get it started just a little, then back off to allow the broach to straighten itself up before proceeding. If the ram isn't perfectly flat or your parallels underneath perfectly flat, the broach will tend to bend or flex a little. It doesn't have so much to do with the broach making a crooked slot, but for what a broach cost, you don't want to flex it to the point that it may snap.
With the set that you have, did it come with shims or is it a one shot broach? All of the sets that I have used, you have to run the broach through twice. The first time you use just the broach and bushing, and the second time, there is an appropriate thickness "L" shaped shim that you drop in and broach it again. This makes the correct depth slot in your part. Just out of curiosity, what brand is your set and what did it cost? I wouldn't mind having a set for here at home.