Depending on what sort of surface finish / accuracy you need in the hole, and how hard your rubber pieces are, I would go with grinding using a burr of some sort. My experience with drill bits in soft material is that they grab and pull the material, leading to really poor results. A carbide burr, used with a slow feed (the goal being to not compress / deform the material) should give you a nice result.
If it's hockey-puck hard, any woodworking tools should get it done, as mentioned above.
All of the above based on two projects trying to make shaped rollers out of soft-ish urethane.