Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
Anyone else doing a datacenter in the garage?
- Thread starter Digital Spaceport
- Start date
pizza
Well-known member
nice power bill
i use mostly consumer/prosumer hw (but i do have rack envy looking at that)
i use mostly consumer/prosumer hw (but i do have rack envy looking at that)
Augus7us
Well-known member
Are you mining bitcoins?
I bought a half rack for some audio gear and maybe some nas storage.
I bought a half rack for some audio gear and maybe some nas storage.
jkeyser14
Well-known member
Everything I need at home fits into a 4U space.
good use for a standard heat pump (not a mini-split) and a Honeywell JADE economizer. gotta get that free cooling to keep costs down.
Might need more than one HP or AC unit.
A few years back I was in a guys house and he had a Bitcoin mining operation. We added up the amps on his server and came up with 7-1/2 tons of cooling! He said he wasn't surpise because in the winter he opened the door for the room and set a few fans out and could heat his very large home with the rejected heat until it got below 0F.
A few years back I was in a guys house and he had a Bitcoin mining operation. We added up the amps on his server and came up with 7-1/2 tons of cooling! He said he wasn't surpise because in the winter he opened the door for the room and set a few fans out and could heat his very large home with the rejected heat until it got below 0F.
58Yeoman
Well-known member
Wind turbines are the answer.
PassnThru
Well-known member
You need to expand on how you are utilizing them - if it's for personal use then no - it's unlikely anyone is doing that as a single PC is more than adequate.
How many clients do you have? What's your network capacity to the outside world?
How many clients do you have? What's your network capacity to the outside world?
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
I do Chia crypto farming and plotting as a service primarily on it. Proxmox cluster for a few of the servers for some general development work.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
Chia and some other more "compute" friendly crypto stuff. Also some HA clustering on Proxmox.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
Dell Fresh Air Compliant servers are on the flanks, and they dont all run all the time. Portable AC can be directed right at the center and keep the spinning hdds down to about 40c easy. I have radiant barrier in the attic...but I am also here to learn more about insulation techniques for the garage
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
Yeah those asics are like 3.25KW each now man. I have a total racked power budget of under 14KW and dont run everything all the time. Those quad socket servers dont even stay on in "powered off" state since they eat 35w "off". I fire them up for special uses only. I found this place while looking for A/C and insulation advice for garages...and frankly I love this forum already. So many amazing workbenches.
That Honeywell JADE economizer mentioned above has me very interested! However I was thinking of a mini split.
JADE™ W7220 Economizer Logic Module | Room & Zone Controllers | Building Controls | Controllers & Control Panels | Room & Zone Controllers | Honeywell Building Solutions
Learn all about the Honeywell BMS NA JADE™ W7220 Economizer Logic Module. Click to find product details, documentation, ordering info and more.
TLDR: if it's nicer outside than inside, it'll open a damper and turn on an exhaust fan, instead of running your AC. factors in wet bulb temp since dehumidifying can cost more than cooling, depending.
When it's economizing, you only pay for the fan power instead of fan + compressor power (or less compressor power if it's in combo mode.) All automatic, no manual/seasonal changeovers required.
I just did one at work (5 ton). I also wanted the 1000CFM+ of fresh air it will provide, which is part of why it's built the way it is. ODU is an inverter 5 ton with standard Y1/Y2 O/B wiring, so it dials itself back as the discharge air temp falls down. great at dehumidifying.
Chilltrix has a chilled-water minisplit offering with an integrated free cooling coil intended for your application.
Air-to-Water Heat Pump Manufacturer | AWHPs, Buffer Tanks, Fan Coils | Chiltrix
Manufacturer of monobloc air to water heat pumps, most efficient, worlds record, for radiant heating, heat pump water heating,fan coil units, buffer tanks, pricing, ATWHP, AWHP, research, compare, design, cost, buy - air to water heat pumps, compare performance specifications, system designs
www.chiltrix.com
PassnThru
Well-known member
Good luck. Don't forget to track all of your costs including building out and maintaining the infrastructure as well as the monthly utility cost. And, as always, factor in volatility in the product.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
The UPS logging makes it easy. May need added logging if I get an A/C dedicated in there. The wall plugged unit I don't track yet but it's about time to start. Do you know of any AC that does logging of power usage?
Metal-Marc
Well-known member
Yeah, I have three racks of audio gear in my living room.
mcbane
Well-known member
If you do metalwork, be sure to use compressed air to blow filings and grinder dust off the circuit boards periodically. Most warranties void if more than 1/4” layer of conductive grit and dust on the boards.
Miss the Pontiacs
Well-known member
I haven’t put any racks in at this point. But I did wire my garage with 6 pr, Coax, 4 pair telco cable and a 12 strand of fibre all buried. Not that I will ever use it all but had it laying around from some other jobs and decided to use it rather than have it take up space.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
I do all the wood and metal work outside and try to vacuum at least 1 time a week with the bristles on the shop vac. The insides stay pretty clean as long as I keep the garage closed.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
I like your style! 10/40gb fiber gear now is so cheap if you ever decide to tap that you will be happy you planned it out. I have OM2 ran in the house to all the rooms and thought it was overkill when I did it but it's now in use.
Grant Gunderson
Well-known member
No way I’d put mine in the garage. Too much dust from wood and metal working projects in there.
When we purchased our current home, I turned the small home office closet into a climate controlled space with dedicated air control for the heat and placed my rack with both computer gear and all of the AV stuff for the whole house in it. It’s totally quiet in my office with the closet door closed and it keeps all of the heat from the hard drives out of the office. 150TB of drives produce a lot of heat! Plus it’s a great spot to hide my 5 year olds Birth day presents. During work season where my sons presents currently are is where I charge camera batteries.
When we purchased our current home, I turned the small home office closet into a climate controlled space with dedicated air control for the heat and placed my rack with both computer gear and all of the AV stuff for the whole house in it. It’s totally quiet in my office with the closet door closed and it keeps all of the heat from the hard drives out of the office. 150TB of drives produce a lot of heat! Plus it’s a great spot to hide my 5 year olds Birth day presents. During work season where my sons presents currently are is where I charge camera batteries.
rayra
Well-known member
Nyet, same reasons as above. Too much sawdust, too much heat.
I'll eventually put a small rack cabinet high in my office closet to host my POE security camera system and NAS
I'll eventually put a small rack cabinet high in my office closet to host my POE security camera system and NAS
rooster59
Well-known member
You can't have any humidity that condenses on equipment. The humidity specs on electronics are "non-condensing". If the garage is cool and you open the doors to hot wet air like after a summer rain, the moisture in the air is going to condense on everything inside the garage.
same problem for anyone with tools. the electronics are at least self heating, which is somewhat mitigating.
OP has some Dell Fresh Air stuff, which is pretty happy at 100F. most gear will do just fine at up to 80F, so it should be pretty easy to avoid over-cooling and running into that issue.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
I open the doors very little currently and am looking for better ways to air seal around them currently. I usually only open them up if its night and humidity is under dew point. I do have a dehumidifier that I can run during the fall when its added heat isnt a huge issue but I should work out a "pipe up" into the attic exhaust for that so I can run it more often.
rlitman
Well-known member
I would not suggest ANY standard equipment for data center cooling. And a mini-split is the second worst option of all, with portable ACs being by far the very worst of the worst.
Except that when it's running, the heat from the power consumption will prevent condensation. Not only that, but if you review your psychrometric tables, you'll see how the heating from the equipment actually "dehumidifies" the space. The absolute humidity will stay the same, but the RH will drop as the temperature rises.
Still, high humidity destroys air filled hard drives, and that happens will below condensation levels. Either stick with helium filled drives or SSD if you're not controlling humidity. Low humidity in the winter has it's own risks.
Tying together humidity and my first quote above, the issue with standard cooling equipment is that any equipment designed for comfort cooling is trying to strike a balance between latent and sensible cooling, where latent cooling includes dehumidification. If your space is properly sealed, you only require sensible cooling, since electronics do you put out moisture. The way to achieve this is with very high airflow rates over the air-handler coils. Mini-splits aren't designed to work like this. Neither is most residential cooling, but there are systems where you can speed up the fan. Without that, you'll be throwing money out the window.
jfleisher
Well-known member
jfleisher
Well-known member
My data center utilizes a "pod" design, containing the hot air in the center aisles. Average temperature outside the pods is 74 degrees F. Inside the pods, the temperature ranges from 100-115 degrees F.
dcg9381
Well-known member
I've run servers in the garage/shop before. Main issue is dust (rather than temperature). I gave up running that many servers years ago though, VMs are much easier (assuming you're not mining).
When we completed the house, I setup separate AV and network closets. Both have AC drops. I also installed bathroom vent fans so I can push heat outside, if necessary. Both closets have power backup.
I run 9 racks of servers for work (in a data center) - same deal, alternate hot/cold isles.
When we completed the house, I setup separate AV and network closets. Both have AC drops. I also installed bathroom vent fans so I can push heat outside, if necessary. Both closets have power backup.
I run 9 racks of servers for work (in a data center) - same deal, alternate hot/cold isles.
86turbodsl
Well-known member
Yeah, a single rack in the basement. I stopped running so many computers though. Proxmox/VMs for everything on a single box. My power bill is already high enough.
I'm gonna disagree with most of what you said there. as a former network engineer in a $B business, standard equipment works fine, because it's so simple to control.
minisplits work GREAT (they auto-size to the load), especially if they are a model with power-fail recovery. set them to cool 78F and aim the discharge at the equipment.
The only equipment I hate with a fiery, fiery passion are portable ACs during construction. GCs love to "provide cooling" by tenting your stuff and putting a portable single-hose AC in said tent. when I try to get it through their heads, I've found that asking "you ever use a shop vac?" then pointing out what they just gave me is a shop vac with my equipment inside is often the only way to get the point across.
please note the context here is OP's garage or IDF/MDF, not a 4+ figure kW DC.
Preach! I've got over 16 VMs on a single box. including two "gaming" computers with real GPUs. any time I want to try something new it's super convenient. also snapshots are great for hitting undo on a "lemme try this real quick".
ArcReactorKC
Well-known member
I know this thread is a little long in the tooth but I am about to start migrating from having multiple dedicated systems to a real rack and a fairly high end 2u system running at least 10 vm's. Do you have any suggestions on best hardware solutions?
I am lightly familiar with VMware esxi but I bet I'm about to get real familiar. I need to have multiple ubuntu vm's and multiple windows server running. I have yet to do this on a single box so I think it's going to be quite interesting. That's not even including migrating from microsoft 365
i'm about to test out XCP-NG instead of ESXi. too many limitations in the free version of ESXi.
if you're going to run ESXi, check the license limitations before you spec out. the current "Free" version is single socket, 32 core max. might be a problem if you have dual socket, 10 core each.
I'm a big fan of https://www.theserverstore.com
i've noticed their ram prices are insanely cheap compared to buying aftermarket. I just got a machine with 256GB ram in it for under $30/stick. the price i got quoted elsewhere was $60ea, and the list price was $93.
as for what you want, most chips are TDP limited, so if you want a lot of slow cores, or a few fast cores, that's kinda up to you. depends on what you expect to run for VMs/what kind of single threaded performance you need.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
Some updates. Racked and running now pulling around 8KW/h. Heat was going to be a bit of an issue, but I did a drill and fill and attic insulation job. R60 up there. Also super quiet now. Added a 24k but window unit. It's not bad in here. 101F outside, 84 in here. Window units not optimal in placement however. Def some work to be done on the window are front. Also need to finish the plugs up here from the drill n fill. That's a totally messy job. Now for some pics.
Attachments
dcg9381
Well-known member
Are you making any money with that? Those cores aren't optimized for crypto. I assume the HA cluster is for "fun" and not as a leased service?
One of my employees is doing crypto at home, but he's buying GPUs (high dollar).. I think he was clearing $2k/mo (after power costs) when crypto was high, as it's 50% of that now, I'd guess $1k a month. He's got $60k invested in HW though... All of it is outside.
One of my employees is doing crypto at home, but he's buying GPUs (high dollar).. I think he was clearing $2k/mo (after power costs) when crypto was high, as it's 50% of that now, I'd guess $1k a month. He's got $60k invested in HW though... All of it is outside.
ArcReactorKC
Well-known member
After your response I started doing some research, I am currently testing out proxmox on a cheap box setup. It took less than three hours from hardware assembly, to having pi-hole running in an ubuntu container. For free I really can't complain.
Now it's time to get some real hardware.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
If you are thinking in terms of monero, no they are not optimized at all. I do a crypto called Chia. I run a service producing chia plots and sell hdd's in bulks. It's a hard drive crypto so that all makes sense. These quad socket R920s and R930s are absolute beasts for creating plots. It's one of the best whole system benchmarks out there. I can get them made in around 4:30 seconds on the r930's. 30 seconds off from the record and way faster then most ppl. I sell hdd batches and if ppl want them plotted I charge extra for that. I can do it faster and cheaper then they can w/o massive investment. Wrote 125TB of them yesterday, net profits not bad at all. Would take most ppl a 1.5-2 months to do that with a single desktop with a top end processor. The R920's while rather old are crazy powerful with their 8895v2's. They are a top 50 in multisocket systems on passmark. Have not hit the 8890v4's yet on there but those should get some good results also. https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E7-8895+v2+@+2.80GHz&id=3514&cpuCount=4
GPU's are tuff to be in right now with the Merge hitting soon. I had a pretty large GPU farm back from '14-'19 but prefer the lower electric hit from this. Those GPU's ran 70 amps, this all together maxed hits around 45.
OP
Digital Spaceport
Active member
- Joined
- May 20, 2022
- Messages
- 43
Proxmox is awesome. If you can get a dual socket dell, they have IPMI tunable fans so they dont annoy the **** out of you. How many VM's are you thinking of running?