Anyone else doing a datacenter in the garage?

[u2slow]

tldr;

I do want to move my server rack to the shop eventually - and free up the extra bedroom completely - but I have to build the enclosed mezzanine space first. No sense dragging computer equipment out there now to suffer the temperature swings and filth.

 

[niget2002]

My little rack. Just doing typical server stuff... plex, pi.hole, nfs server, backups, etc etc.

I do occasionally run a handbrake cluster for transcoding videos.

It's been running in my shop for a few years now. I put a 20"x20" AC filter in front of it to trap most dust/debris before it goes into the servers.

 

[niget2002]

Oh... My servers cost $15/month/ea in electricity. The 1U is usually turned off unless I need the extra horsepower for something.

 

[ArcReactorKC]

Proxmox is awesome. If you can get a dual socket dell, they have IPMI tunable fans so they dont annoy the **** out of you. How many VM's are you thinking of running?

In the end I will probably have 8-10 full VM's and a few containers. I've currently got my mail server, web server, inhouse data, public data, customer accessible data, regular home data files, and the kids nonsense data files. All of those are currently separate boxes. It's terribly inneficient but I don't have a single machine capable of handling the data needs much less the horsepower to run it all at once. I'm hoping I can conglomerate all of that into a single machine for the VM's and routing needs with a separate in house large NAS for the actual storage. This project started out super simple but it's really getting some scope creep.

 

[Digital Spaceport]

In the end I will probably have 8-10 full VM's and a few containers. I've currently got my mail server, web server, inhouse data, public data, customer accessible data, regular home data files, and the kids nonsense data files. All of those are currently separate boxes. It's terribly inneficient but I don't have a single machine capable of handling the data needs much less the horsepower to run it all at once. I'm hoping I can conglomerate all of that into a single machine for the VM's and routing needs with a separate in house large NAS for the actual storage. This project started out super simple but it's really getting some scope creep.

I'm going to suggest something a bit more ez/reliable out of the box which is unRAID. You sound like low VM count which unRAID can do also. It's got an excellent easy docker "app store" interface. Data protection via parity drives. Easy to manage storage configurations and if you want to run publicly facing stuff just sit it behind an nginx reverse proxy. App does exist for that also, but I'd recommend a cloud vps for 5/mo from DO as a superior option or cloudflare. You can run all this off a Dell R720XD which has 2x 2.5" bays in the back for an SSD cache pool, 12x 3.5" bays in the front to grow a nice storage array, and a lot of slots for cheap ecc ddr3 and performant E5-2697'v2 processors for pretty cheap.

I love proxmox, but I use it for pure power on the VM side and have external datastores for a lot of the shares...more complexity. It is not an awesome storage management OS out of the box without some specific configurations and data loss is a real potential if you "oops"
some things.

 

[ArcReactorKC]

I'm going to suggest something a bit more ez/reliable out of the box which is unRAID. You sound like low VM count which unRAID can do also. It's got an excellent easy docker "app store" interface. Data protection via parity drives. Easy to manage storage configurations and if you want to run publicly facing stuff just sit it behind an nginx reverse proxy. App does exist for that also, but I'd recommend a cloud vps for 5/mo from DO as a superior option or cloudflare. You can run all this off a Dell R720XD which has 2x 2.5" bays in the back for an SSD cache pool, 12x 3.5" bays in the front to grow a nice storage array, and a lot of slots for cheap ecc ddr3 and performant E5-2697'v2 processors for pretty cheap.

I love proxmox, but I use it for pure power on the VM side and have external datastores for a lot of the shares...more complexity. It is not am awesome storage management OS out of the box without some specific configurations and data loss is a real potential if you "oops"
some things.

unRaid is definitely next on my list of "maybes"
I'm currently setting up winserv2019 in the new proxmox tester here. We will see how some of this goes. I think you are correct though on unraids better data handling abilities.

 

[niget2002]

Take a loot on amazon. Amazon sells refurbed boxes for pretty cheap. My 2u Dell R510 was only $800 and came with 25TB of sata storage, 2 CPUs, and 128GB of RAM.

Looks like it's $900 now, but still a decent deal. It comes pre-loaded with windows server on it, but I installed ESXI on a USB key and put it on the motherboard.

https://www.amazon.com/gp/product/B07TJ1P3Q7/?tag=atomicindus08-20

 

[dcg9381]

Ebay too.. The market for off-warranty servers is incredibly low.
I have lots of friends that were running individual 1-2U machines for various technical things (mail, DNS, file share, whatever) - due to power requirements, I know I'm not doing that again... Stuff it all into a single machine and run VMs!

 

[u3b3rg33k]

Ebay too.. The market for off-warranty servers is incredibly low.
I have lots of friends that were running individual 1-2U machines for various technical things (mail, DNS, file share, whatever) - due to power requirements, I know I'm not doing that again... Stuff it all into a single machine and run VMs!

not to mention how much slicker it is to try something out with a VM. snapshotting is a massive time saver! so is making a template of a machine you built and deploying a bunch of it.

 

[dcg9381]

not to mention how much slicker it is to try something out with a VM. snapshotting is a massive time saver! so is making a template of a machine you built and deploying a bunch of it.

For my job, we've gone 100% virtual, although we may run in a 1:1 ratio of VM to bare metal. Being able to snap-shot something and immediately un-F' it has been amazing. VMWare ain't cheap though.

 

[Digital Spaceport]

Take a loot on amazon. Amazon sells refurbed boxes for pretty cheap. My 2u Dell R510 was only $800 and came with 25TB of sata storage, 2 CPUs, and 128GB of RAM.

Looks like it's $900 now, but still a decent deal. It comes pre-loaded with windows server on it, but I installed ESXI on a USB key and put it on the motherboard.

https://www.amazon.com/gp/product/B07TJ1P3Q7/?tag=atomicindus08-20

I wouldn't go lower then a 12th gen Dell currently. The wattage drop idle states + double the cores was a huge boon for Sandy/Ivy chips over the prior gen. My R720XD can idle at 90w with top end chips, but my older r710 couldnt idle under 210 with top end chips.

 

[niget2002]

I wouldn't go lower then a 12th gen Dell currently. The wattage drop idle states + double the cores was a huge boon for Sandy/Ivy chips over the prior gen. My R720XD can idle at 90w with top end chips, but my older r710 couldnt idle under 210 with top end chips.

That's fair. I bought this one over 2 years ago.

Edit... My memory is pretty good. Just looked. I bought it Nov 2, 2019... So coming up on 3 years.

 

[shade]

Plan this out - if you think you will need more vms over time grab 2 similar servers. Then you can use vcenter and migrate servers from one host to another in real time (vmotion). Primarily in case of downtime or failure.
I have 40 vms on a single server in my home datacenter so there is no reason to not pack these things are densely as possible.
Also dont run a home mail server or web server. Put it in the cloud or decom it
Your asking for major trouble (getting into web and email servers is relatively easy. I work in cybersecurity for a living)

 

[u3b3rg33k]

Plan this out - if you think you will need more vms over time grab 2 similar servers. Then you can use vcenter and migrate servers from one host to another in real time (vmotion). Primarily in case of downtime or failure.
I have 40 vms on a single server in my home datacenter so there is no reason to not pack these things are densely as possible.
Also dont run a home mail server or web server. Put it in the cloud or decom it
Your asking for major trouble (getting into web and email servers is relatively easy. I work in cybersecurity for a living)

I used to run my own mailserver. now I use mxroute. I can't beat their prices or their uptime unless I assign zero value to my time (or power bill), their tech support is great, and they give you a real AAR/IRR after an outage/event.

50 GB Storage

Unlimited Domains

Unlimited Email Accounts
$65 /year

 

[MattRMagnum]

With our move, my office is temporarily (for the next 6-12 months) in my house. After that it'll be relocated, along with my server rack, to the loft in our new garage. I think I worry less about dust/grime than most? I have filters on my server racks, but I found that even when stored in our house, they required regular cleaning from dust, so I suspect my garage won't be much better or worse, unless I've an engine fire or something (in which case, I likely have larger problems).

I don't run much. I've a NAS I use for backups/data storage, and a box that runs a few vm's (media server and similar). Eventually I'll convert the low-quality security cameras on the house to be PoE ones that connect back to the NAS/virtualization server, but that's money and future plans.

 

[inphx]

nice tip on mxroute. we do on premises smartermail install for secondary domains.

I built in a server/electronics lab into my garagemahol. Now i am permitted for 16kw of off grid solar so that will let me more liberally cool. All these servers are for offsite backup or pet projects. We have a datacenter rack for real stuff.

 

[shade]

nice tip on mxroute. we do on premises smartermail install for secondary domains.

I built in a server/electronics lab into my garagemahol. Now i am permitted for 16kw of off grid solar so that will let me more liberally cool. All these servers are for offsite backup or pet projects. We have a datacenter rack for real stuff.

More pictures or we consider this all fake

 

[ArcReactorKC]

I'm going to suggest something a bit more ez/reliable out of the box which is unRAID. You sound like low VM count which unRAID can do also. It's got an excellent easy docker "app store" interface. Data protection via parity drives. Easy to manage storage configurations and if you want to run publicly facing stuff just sit it behind an nginx reverse proxy. App does exist for that also, but I'd recommend a cloud vps for 5/mo from DO as a superior option or cloudflare. You can run all this off a Dell R720XD which has 2x 2.5" bays in the back for an SSD cache pool, 12x 3.5" bays in the front to grow a nice storage array, and a lot of slots for cheap ecc ddr3 and performant E5-2697'v2 processors for pretty cheap.

I love proxmox, but I use it for pure power on the VM side and have external datastores for a lot of the shares...more complexity. It is not an awesome storage management OS out of the box without some specific configurations and data loss is a real potential if you "oops"
some things.

I just wanted to come back to this and say unRaid has been very smooth install and migration. Once I get my new hardware it'll also be super easy to transfer the whole system. Big thumbs up on the suggestion.

 

[Outlander]

My wee Homelab located in the basement workshop at home. Mostly tinkering for my own gratification and knowledge, so I can hold my own with my team of IT professionals. I'm running a Synology NAS, Proxmox, TrueNAS, Pi-hole as well as a few other services like Grafana on a RPI not yet installed in the rack (needs a rebuild to 64-bit).

This is not the latest photo (I am not at home to get an update).


My 'mad scientist lab' where I have breadboards with RPi, Arduino and related stuff is in my office, on a separate table.

Picture was taken while I was cleaning up and rewiring. I have a couple of Homelab services running here, including temp/humidity monitoring. I use an ESP8266 board with a sensor on my 3D printer to keep an eye on temp/humidity, may add one to the Homelab rack as well. That said my 2 servers are actually beefed up Dell workstations.

 

[Digital Spaceport]

My wee Homelab located in the basement workshop at home. Mostly tinkering for my own gratification and knowledge, so I can hold my own with my team of IT professionals. I'm running a Synology NAS, Proxmox, TrueNAS, Pi-hole as well as a few other services like Grafana on a RPI not yet installed in the rack (needs a rebuild to 64-bit).

This is not the latest photo (I am not at home to get an update).


My 'mad scientist lab' where I have breadboards with RPi, Arduino and related stuff is in my office, on a separate table.

Picture was taken while I was cleaning up and rewiring. I have a couple of Homelab services running here, including temp/humidity monitoring. I use an ESP8266 board with a sensor on my 3D printer to keep an eye on temp/humidity, may add one to the Homelab rack as well. That said my 2 servers are actually beefed up Dell workstations.

Dell and HP Z workstations are surprisingly very good if you don't need oodles of RAM

 

[macnugget]

Next week I'm trenching fiber from my house to the workshop so that I can have both buildings on the same network. The previous owner of my property just paid for two separate internet connections and I don't want to be stuck having to route out to the world and back in just to ping my garage.

I plan to have a small deployment in the workshop so I can use it as an "offsite" backup location for my NAS in my main house.

 

[Digital Spaceport]

Next week I'm trenching fiber from my house to the workshop so that I can have both buildings on the same network. The previous owner of my property just paid for two separate internet connections and I don't want to be stuck having to route out to the world and back in just to ping my garage.

I plan to have a small deployment in the workshop so I can use it as an "offsite" backup location for my NAS in my main house.

10g or 40g?

 

[macnugget]

10g or 40g?

Just 10g, I'm runing UniFi switches and that's what they support.

 

[rayra]

I might of missed it but don't see any words about what the OP is doing for dust control / intrusion into the rack environment.
My own garage is so hot and dusty I'm cringing just hanging a 24" flatscreen over my workbench. I'm thinking about putting an arched plexiglass shield spaced above its top vents just to try and keep sawdust from piling down into it.

Maybe ducting the cooled air right into the rack enclosure for both direct cooling but also to overpressurize the cabinet to prevent dust intrusion?

 

[Digital Spaceport]

Just 10g, I'm runing UniFi switches and that's what they support.

thats still a shitload of BW at 10Gb

 

[Digital Spaceport]

I might of missed it but don't see any words about what the OP is doing for dust control / intrusion into the rack environment.
My own garage is so hot and dusty I'm cringing just hanging a 24" flatscreen over my workbench. I'm thinking about putting an arched plexiglass shield spaced above its top vents just to try and keep sawdust from piling down into it.

Maybe ducting the cooled air right into the rack enclosure for both direct cooling but also to overpressurize the cabinet to prevent dust intrusion?

So what I do is an active strategy that is 2 part. 1st I have a very decent air seal. I accomplished this with normal blue painters tape. I know but yeah it works great. Its also fast, cheap and not permanent. I dont use the garage doors. Layers of it work wonders.

2nd part. Fine small micron filter on the shop vac and a lot of active vacuuming. Grills, surfaces, tops of things. Dust is magical, it just happens no matter what. I primary only utilize Dell Fresh Air compliant builds for active running machines. These things were engineered for harsh. Like Iraq harsh. To date, have had zero server issues. This is years in, granted its recently gotten bigger then ever.

 

[Outlander]

Dell and HP Z workstations are surprisingly very good if you don't need oodles of RAM

Yes, I'm a bit of a Dell fan-boy due to work, but I was contemplating the HP workstation as well. Some of the higher end models have dual CPU and more room for storage.

The Dell T1700 will take 32gb while the T3600 will take 64 gb (I have 16). This is probably "oodles" for me when maxed out.

 

[u3b3rg33k]

So what I do is an active strategy that is 2 part. 1st I have a very decent air seal. I accomplished this with normal blue painters tape. I know but yeah it works great. Its also fast, cheap and not permanent. I dont use the garage doors. Layers of it work wonders.

2nd part. Fine small micron filter on the shop vac and a lot of active vacuuming. Grills, surfaces, tops of things. Dust is magical, it just happens no matter what. I primary only utilize Dell Fresh Air compliant builds for active running machines. These things were engineered for harsh. Like Iraq harsh. To date, have had zero server issues. This is years in, granted its recently gotten bigger then ever.

you could probably do well with a dry bulb economizer and good filtration (merv 11+), just run OA. don't pay to run the compressor.

 

[inphx]

"More pictures or we consider this all fake"

My pants are on fire...

 

[jeffmoss26]

I have a 1950s vintage sound rack in the basement. It has a 24 port switch, patch panel, audio input panel, mixer/amp, 2 servers, wireless AP, power strip, small UPS. The Cisco switch serves as a shelf for padlocks on the back side of the rack.
Modem/router is upstairs, I ran a cable outside years ago to feed the shop. I ran cabling all around the basement in high school/college and still use it. Phone systems are/were a hobby so I have a lot of older systems hanging on the wall
My bench is next to the rack, the last 5 ports of the patch panel are an extension off each phone system so I could have a couple phones on the bench before I had 2 monitors.

 

[ArcReactorKC]

This thread has been dormant for too long.
Time for another opinion based question.

I'm moving away from my all in one **** asus setup. Currently running the Asus AIMesh network setup.
I think I want to move to a hardware firewall, router, a larger ethernet switch, and then a mesh AP network.
What does everybody have for these devices?

 

[pizza]

I think I want to move to a hardware firewall, router, a larger ethernet switch, and then a mesh AP network.
What does everybody have for these devices?

generally, unifi or cisco

 

[mogandave]

Maybe there are some other ppl running some racks of servers in their garage?

Servers with racks count?

 

[highdeserthacker]

This thread has been dormant for too long.
Time for another opinion based question.

I'm moving away from my all in one **** asus setup. Currently running the Asus AIMesh network setup.
I think I want to move to a hardware firewall, router, a larger ethernet switch, and then a mesh AP network.
What does everybody have for these devices?

I run Ubiquiti UniFi for wifi mesh, and just an EdgeRouter for firewall and tunnel to another home out of state. It's been very reliable.
One Intel NUC running Proxmox on ZFS for all the VMs (Plex, Pihole, Ubuntu Servers, Home Assistant, etc). Maybe 40 watts.
One 16 TB NAS for all the storage.

 

[fitter30]

If humidity is a problem install a vapor barrier high rh goes to low rh.

 

[macnugget]

I also recommend UniFi as a good "prosumer" solution, but I don't recommend mesh for WiFi if there's any way at all to run ethernet to the access points. Mesh WiFi is absolutely better than a single access point, but there are a lot of compromises with it. You're always going to be better off running cables to the access points if at all possible.

 

[theoldwizard1]

I ran a medium size "data center" in the 90s-early 2000s. You could easily fit at least 6 tennis courts in there ! A/C units were the size of about 4 double door residential refrigerators. We had 4 of them.

Learned about 3 phase wiring (208 Wye) by adding on to existing distribution panels.

 

[u3b3rg33k]

I ran a medium size "data center" in the 90s-early 2000s. You could easily fit at least 6 tennis courts in there ! A/C units were the size of about 4 double door residential refrigerators. We had 4 of them.

Learned about 3 phase wiring (208 Wye) by adding on to existing distribution panels.

I did some work in a place that had a late 80s/early 90s DC. the generator they had outside was probably one of the largest units i've ever seen. the entire DC got consolidated down to a single rack by 2015. generator got sold to some mining operation as it was pre-emissions.

 

[niget2002]

I'll 3rd or 4th the unifi recommendation. I've been using their stuff for quite some time and have upgraded some of the components as the bandwidth to my house increased.

I went with the Dream Pro (mostly for the built-in DVR that I'm still not using :/ ). Then I have a separate switch for the POE out to the APs. I have a second switch in the shop for the servers and the AP out there. I just have the single cat-6 uplink from the shop to the house.

I run all my IOT stuff on a separate vlan and essid than the rest of the house.

 

[Jawn]

I'm a fan of Mikrotik for routing/switching, but they've got a bit of a learning curve for setup. I also tried one bit of their wireless gear, and it was a bit of a hair pulling experience getting that set up to be centrally managed by the switch. The wifi hardware also proved less than reliable. Maybe I got a dud. But I went back to using my old Unifi UAP in the house.