I would never remove it, unless you plan on putting another one it later down the road, as you said you may not use it this year much, but trust me your kids will start using it when hanging out with friends. I was for removing mine about 5 years ago the wife talked me out of it. Now Im so Glad...