They have to set up delivery or load and unload the drywall, set-up, waste time talking to homeowner after showing up, breaks, lunch, clean-up, load waste and haul somewhere to dump (doesn't just disappear when they leave your place), etc.
It's not just the time on site they charge for. Like...