The conductors will need to be in conduit from the panel to the meter if concealed in the wall.
I'm no expert on the NEC, but why couldn't the panel and meter be bolted together, sandwhiching the sheating and siding? Then you would just have a short bushing between the two.
Lastly, In my neck...