X2 on this....Just go to a larger driven pulley. Also look at how things are mounted, you may want some vibration damping mounts.

Beat me to it for that suggestion.
When I installed mine, I let the whole assembly float on sponge rubber pads to minimize vibration transmission to the ceiling joists.
If the current “high“ speed setting moves plenty of air you could install a bigger pulley on the fan blades or a smaller pulley on the motor and that will give you a lower speed and probably less noise. But of course, that will reduce the CFM of your fan.
BTW, do you have plenty of square inches of exhaust area in your attic? That affects the CFM.
I had to add an extra huge gable vent when I put mine in. I move 8000 CFM with mine.