I prefer the look of drywall, but depending upon usage a plywood wall can be much more useful and will hold up to the errant ladder leg or 2x4 that swings against it.
However, it really depends upon what YOU want out of your space. Drywall can easily be made to look finished and attractive. If you get a few extra holes where you don't want them, drywall can be patched and painted quickly and effortlessly.
Plywood has advantages of being able to pound a nail or drive a screw almost anywhere, but some people act as if you can just mount cabinets or heavy items anywhere - and that is only possible if you opt for a thicker material. I would not mount heavy items to 1/4" plywood because I'd fear screws or nails could pull out. If I had 3/4" material I'm sure it could handle almost anything but the heaviest items, in which case you would still want to find a stud.
Of course plywood makes treating the holes and seams more difficult. If you want a finished look, you'll need to fill all the nail/screw holes with filler and treat the seams. Some use caulk, others use batten boards - but neither of those two options will ever look as "finished" as a properly taped drywall seam, so it is just a matter of what you want for your area.
I'd shy away from OSB and MDF - yes they are less expensive, but they don't hold fasteners as well as plywood, OSB doesn't look as nice even after painting (you tend to see some of the strands), and MDF doesn't hold up well to moisture (which almost all shops/garages deal with in some capacity whether it be due to spashes from a shop sink, or simply humidity in the air). I've seen a lot of warped OSB and MDF over the years almost always due to humidity, so even if you do use them on the walls I'd shy away from them for a ceiling.
I once new a contractor who build his home with 3/4" plywood walls which were then covered with 1/2" drywall glued and screwed on top. His logic was he never wanted to worry about hiting a stud when hanging a picture... but it seemed like a lot of expense for such a small benefit. Today with a quality stud finder, locating a stud isn't a challenge, and most larger/heavier items span multiple studs anyway, so I personally would opt for drywall over plywood just for aesthetics.
All of that said, I've always been a fan of having galvanized "wavy" steel panels on the walls of a shop at least for the bottom 4 feet or so. It holds up better to wear and tear, is easy to clean, no risk of fire, no concern over moisture, and it just has a certain classic look to it. It isn't for everyone, but given the choice I'd have primarily drywall with steel panels covering the bottom 4 feet. Seems like the best of both worlds.