VFDs can operate in a number of different modes - V/Hz mode and Vector mode are very common. V/Hz is a "dumb" mode, where the VFD just outputs a certain frequency and voltage, and the motor is free to do whatever it wants. In vector mode, the drive is constantly looking at the current motor speed and makes corrections to try and keep it at the target speed.
Vector mode can work in both "open loop" sensorless configuration, and "closed loop" configuration with a speed sensor/shaft encoder. In open loop mode, the VFD relies on feedback from the motor's electrical connections, while in a sensor configuration, it relies on the shaft sensor for all its information.
If it's a closed loop drive, you may have a problem with the shaft encoder/speed sensor. If its open loop, the drive it self may be failing, or you may have something introducing a lot of electrical noise into the circuit.
Also, are you sure it's a VFD, and not a DC motor drive?