Check to see if the saw actually travels in that direction as it is lowered or if there is any slop in the hinge first. If it is tight and the blade just wants to go a different way then the saw travel then my understanding is that it's the blade. My saw cuts very straight normally (I can take 1/64" slivers over 3" angle that measure within a couple thou top and bottom), but just the other day I threw on a cheap hss blade and it wanted to go off to the side right from the start, possibly because the first thing I cut was thin wall. I may not have broken in that blade, which is done by cutting something plenty thick (solid) for the teeth count with lighter than normal down pressure. The teeth on a bandsaw blade are not in a straight line, they're alternating leaning one side and the other. If a manufacturing defect or some mis-use (cutting to thin wall with too course of blade) takes that out you can never get it back.
I've seen different opinions of tension. Some say really tight, some say just a little past keeping it on the wheels. I run mine about as tight as I can get it by hand, but the tension knob isn't huge.