Are you sharpening by hand using a grider and a jig or totally freehand. I have read countless publications that state freehand sharpening is not a practicle approach. To creat a bit that is symetrical a jig must be used. I have a guy that works in my shop that is in his 80's and he hand sharpens with out a jig all the time, he is not producing round holes with the bits, nor does he care as it is just for crude fab and machine mantainance. Not calling saying your wrong, just wondering if you have some insight that i do not on the subject.
You need to find yo' self an old timer.

All joking aside. I learned machining and thus drill sharpening from a 89 year old retired master tool and die maker. He laughed at drill doctors, sharpening jigs and digital measuring equipment. Back in the day, a machinist was expected to know how to properly sharpen a drill bit by hand. Or at least so I am told.
It is one of those things that I could show you in person in 5 minutes, but take 5 pages to try and type out. This guy on youtube is a really good machine shop practice teacher (I think he is actually a retired teacher). He has a really good video on sharpening drill bits.
The key is to NOT twist the drill when sharpening it. That may sound counter intuitive when looking at the cutting edges, but a rocking motion is more appropriate. As others have mentioned purchase a nice drill sharpening gauge. Now when sharpening, what you really care about is symmetry of both cutting edges. It doesn't matter if the angle is 114°, 115°, 120° as long as they are the same. If one side is longer than the other, the drill will cut a larger hole. This can be used to your advantage on certain occasions.
Drill presses and the drilling process I think are the #1 misused and misunderstood machine tool out there. One might say that I am a drill press addict (owned over 6 at one time for various functions). I am far from an expert but I say this based on four common issues that I see.
1.) Lack of speed.
2.) Lack of feed.
3.) Thinking that drill bits produce a round hole
4.) Dull bit
First, SO many people think the solution to their drilling issue is to slow the drill bit down. If you start reading on fab forums across the web everyone thinks they need a 5RPM drill press to drill steel.

When we talk about cutting tools what you care about is SFPM of the cutting face. For general shop use a rough formula can be...
Steel RPM = (4*75)/(Dia of drill bit in inches)
Aluminum RPM = (4*200)/(Dia of drill bit in inches)
For example a 0.5" dia drill bit in steel SHOULD be running at ~600RPM
or in aluminum at ~1600 RPM!!!!
Now the second issue comes into play on non rigid drill presses and nervous operators. If a drill bit chatters it means that you are feeding too SLOW for a given RPM. This will actually destroy the drill bit as it is just rubbing on the work piece, and not cutting. But the problem is that people usually back off when they hear chatter, when instead they should push forward.
A drilling operation is not a precise process. No matter how expensive of a drill bit you have it will never produce a round hole. Hence the presence of reamers. Yes some are better than other, but no drilled hole has a high level of circularity.
Finally, I see so many folks use dull bits because

Most often people will just run out and purchase a new bit, instead of trying to sharpen the one they have. Now if it is less than 1/8" in diameter I usually just chuck it, because those small bits are so cheap in the bulk pack.
One last tip. Always center drill your parts when possible. A drill bit is not designed to center cut well. Unless you get into split point / web thinning sharpening, but we will leave that out for now.
Well enough of my rambling. Go fire up that grinder and sharpen your bits. When I was learning, my teacher stood next to me, as soon as I sharpened a bit, he flattened it on the other grinding wheel and I had to start again. Also make sure your grinding wheel is freshly dressed and true.
The clevleand twist drill company has or had a book out all about drill bit sharpening and geometry. I think it dates from the 1920's, but still a very good book. Let me see if I can find it electronically on my computer.