Everyone wants to grind the floor before epoxy...why,....to ensure a surface that the end product will adhere to.
I'm thinking a broomed surface would more than ensure adherence. Depending on the "broomed-ness" I'd think it would work fine. Of course excessively deep broom groves might prove a challenge to wipe up some spills.
You would likely have a coverage issue too. The squeegee would run over the high ridges and leave the epoxy in the lower valleys giving your floor a funny looking finish I think. The high ridges would be concrete while the rest would be smooth and glossy. Unless you used a lot of epoxy and laid a thick layer. I would vote for grinding it down.
Curing time depends on temps, thickness, how well you followed and mixed the 2 parts for proper proportions. Generally it's walkable the next day, and ready for vehicle traffic in 2 days. Although that is general, I would wait a few days for anything other than stocking foot traffic. I have followed a lot of directions that weren't exactly accurate.