Make the shaft blank (round part) in the lathe, turned, drilled, tapped, etc.
Mount that turned piece in one of these square collet blocks in a 5c collet of appropriate size.
Clamp the square collet block in a mill vise and mill with any appropriate endmill. Set it up against a stop or with a...