Another vote for leaving the scale unless you're looking for an automotive quality finish. I do think it's important to remove as much of the mill oil residue as possible. I do this by "wet" sanding, using rubber gloves (most important), a dilute solution of purple degreaser, and 80 grit waterproof grinding belt cut into short strips. This leaves the mill scale but the abrasive action coupled with the degreaser helps pull out the oils. I consider it done when I can rub my finger across the dry surface and pickup little to no residue.