I generally leave them alone as well. They serve the purpose of consuming other, less desirable insects.
Sure, you might get bit by a spider one day, but they are more likely to scoot away from you than "attack" you.
I kinda feel bad when I kill one accidentally by moving something around in my...