A Grizzly bench top mill will drill holes all day, you just won't have much power or much Z space. You'll need a drill chuck that fits the machine, and a bench top may not be the common R8. Instead it may be an MT spindle, but that's not a big deal. You'll need a drill chuck arbor which is MT on one end and likely JT to fit into the drill chuck. Some chucks like Albrecht have an integral arbor (optional, not all models) which cannot be changed out. If a drill chuck did not come with the machine you'll have to get one, and the arbors are inexpensive and readily available in most configurations from mscdirect.com or use-enco.com. Also places like Traver's, Grizzly, Shars, CDCO, JTS and others. From what I've seen all of Grizzly's tooling is basic import stuff at slightly higher prices.
A mag drill will work, but you likely won't have much spindle travel for deep holes. Some mag drills are set up for annular cutters and won't have a 3 jaw chuck at all. In most cases you can retrofit a chuck, but not always, or parts may be difficult or expensive to source. All in all a lot of work.