I don't think you need to chisel them out unless you are using a masonary product to fill the cracks.
Bondo makes an "all purpose" putty resin that can be used with masonary products. If you can find an epoxy crack repair product, that would work well also.
When you fill the crack, you'll want to force it in as much as possible. Concrete doesn't break smoothly so you're going to have all sorts of spaces in the crack that will mechanically hold the bondo in place.