4" is generally enough, the wall construction will have to be stout enough to support the added floor and the second story walls and so on. usually that means 2x6 framing if current walls are over 8' high..
And if this is to be a legal build, the AHJ will have to permit it, and they may have regs and requirements to add.
best of luck