I still buy a lot of my nuts and bolts from Home Depot, but I buy them by the box now. If I need a few of a specific size, I go ahead and buy a box of 100. That way, I can stock up with the sizes I'm most likely to use in my shop. The 'bulk' box price is significantly cheaper than the individual, or 5-pack, nuts and bolts.
If I need specialty stuff, there is no place like Ace Hardware.
I may end up buying a few 'kits' from one of the big bolt houses online (for a certain price, you get the metal parts storage case and whole assortment of sizes of commonly-used hardware).
In general, though, I'd rather support a local store (which is why I like to buy from Ace, when possible...as close to a local store as I get for hardware). While their 'by the piece' hardware selection is very expensive (but also very extensive), if you go to the counter and ask for a 'bulk' quote, they will call their hardware supplier and get you a 'by the 100' price for whatever you want.
If you live near Sacramento, CA, there is a place called "Blue Collar Hardware." They sell fasteners in bulk at 'by the pound' prices.