Thanks all, one more question. With the logs coming soon, how would I attach that to the basement? Could I just use J hooks? The inside wall attachment seems odd. If anyone has information on that let me know.
Typically, yes, you'd sink J bolts in to the top of the wall before your pour. If you are building a log home, I'd think there would be purpose-built hardware to sink in to the concrete for that. (Longer rods, etc.)
I assume the concern is the look of the foam under the interior of a first floor log wall? If so, I would also go with a synthetic EIFS product, as Red05GT suggested, if you want a masonry look. If not, you can just use wallboard.
Another thing about removing the foam is the surface of the inside is not usually smooth. Most often, it is grooved to hold on to the foam better. I had to remove some foam for my garage door attachments (my house is 100% ICF) and I had a similar experience as matt_i. Once I had all the foam cut off, I had to chisel the grooves off the wall and then use a diamond blade to grind the surface flat. (That was a special kind of dirty hell that I never want to experience again.)