I would choose vinyl.
White and some colors have the color all the way through, so there is no finish to wear off.
Vinyl is a poor conductor of heat and cold, aluminum is a great thermal conductor.
You can use soft scrub to make vinyl windows look new, aluminum will have its finish worn off even if it is anodized.
If your siding is steel, you will have places where steel and aluminum touch, not good. Vinyl is inert to aluminum and steel.
Vinyl is slippery, things tend to not stick. Stuff sticks to painted or anodized aluminum.
Get ones with welded corners on the sashes and frames rather than mechanically fastened units.