80k BTU is definitely overkill for a 20x30 with 50' of shared wall with a conditioned space. I have a detached 24x30 with R13 walls and R22 ceiling and use a 45k BTU. It's adequate. If I had to do it again I'd consider a 60k or 75k BTU unit. I like it warm at 72* minimum and I appreciate a quick temp rise when opening the door.
45k will go as warm as I want I keep it at 60* and turn up to 72* when I'm working, well under an hour to increase the temp and doesn't cycle often. When I got this place it had a 115k BTU furnace in the garage and while it heated up super fast from the 40*-50* I kept it at then, it was less comfortable as it cycled often and I'd feel the temp swing from cold to hot and back. Oversized can work if you open doors a lot, but too oversized is less comfortable and the temp in the space will have large swings and of course lead to shorter heat exchanger lifespan from condensation.