I have had similar problems with casters in the past. It seems to me that cheap casters have too much play in the center pin as you mentioned, but also they don't have enough balls in the bearing. I use the 4 inch, red wheeled casters that I get at Northern Tool. They aren't cheap at $16 each...