Yeah we always burned off the fiber hairs with a propane torch and never had a problem. Pretty quick and easy. It will never be noticed again.
As for the fiber itself it does little to anything for strength. Most fiber used in residential and commercial is designed to be used for shrinkage...