Not all pole barns are created equal, and not all are built with wood contacting ground. Post-frame construction is just a building technique. As with any structure, if it is done the cheap/easy/corner cutting way, it will not last. As with any structure, it if is done the correct way with the correct materials, it will last forever.
My company uses post-frame techniques to build anything from barns, shore homes, through fire stations and medical centers. I can tell you for certain that post-frame construction is a fine choice when done the right way.
edit: Also, as with any style of construction, post-frame CAN be tied into existing structures. You will have no idea afterwards what it looks like underneath the sheetrock and siding etc.
edit2: And to be clear, as with any style of construction, post-frame CAN utilize ANY exterior and interior finishes. Brick, stone, vinyl, steel, hardiboard, shingles, sheetrock, wainscoting etc. etc. etc.