Your choice is largely guided by the size of wheels you’re using and the material you're polishing. Like drilling or turning, the important measure is the speed of the tool at the working surface (usually expressed in surface feet per minute).
For aluminum polishing, something in the range of 5000 to 6000 sfpm is common; for stainless, that gets upped to perhaps 8000-9000 sfpm.
If you’re doing jewelry, for instance, you’ll usually be using small wheels in the 3” to 4” diameter range. Buffers for that work are usually around 3250rpm or so to get into the 3000 sfpm range, which is suitable for those soft metals.
If you’re using large wheels — say, in the 10" or so category — to do things like aluminum manifold polishing, a motor in the 1800rpm range is more appropriate to get the right SFPM.
It's really dependent on just what your hobby is.