This post has been de-listed
It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.
Hello all!
I've been lurking around here for a while and finally have something to ask :) I'm an academic, and I'm about to start a position at a liberal arts college. My focus is on theoretical / computational physics, and a lot of my recent work has used some fun neural-network based tools.
Till now, I've been getting by with cluster access at my previous institution. However, this will eventually disappear. I have some start-up funds to use to set up my research. The funds are time-limited (must be spent within 3 years, otherwise I might just rent time in the cloud). Because of this, I'm planning to put together my own workstation that my students and I can use. Unfortunately, the college doesn't have much in the way of IT support, so I'm putting this all together myself, hence coming to the good people of r/homelab.
I've put together the following system. I'd greatly greatly appreciate if someone can have a quick look over it for anything that is off (before I pull the trigger and waste a bunch of money). I've built computers before, but this is the first "workstation" I've put together (and certainly the first server motherboard needing RDIMMs, etc.).
Much of the work needs FP64 accuracy, hence the 32 GB PCIe V100s (which I'm hoping to pick up used from ebay over the next few months). I'll likely start by running Jupyterhub or something on the workstation, and maybe move to a job scheduler via Bright Cluster Manager a little further down the line.
I know the V100s are passively cooled. I'm planning to use two 3d-printed dual shrouds ( https://www.reddit.com/r/nvidia/comments/17aaeuh/the_nvidia_tesladatacenter_gpu_amateur_info_dump/ ) and two 3d-printed extractors ( https://www.thingiverse.com/thing:807253 ) to create a push-pull configuration through the cards, together with four PWM-controlled ARCTIC S8038-7K 80 mm high-static pressure server fans.
The system won't be in a server room per se - I have a rather large lab space, so I was going to hide it in a corner. I'm hoping to get a script going that will control the PWM for the GPU fans based on the GPU temperatures.
That's enough of a detail dump! Two quick questions:
- Are there any obvious problems with the build below? I've checked everything I could think of from my limited knowledge - e.g. CPU is 200 W, GPUs are 250 W each, the T30 fans are 5 W each, the S8038-7K fans are 6 W each, which comes to 1269 W, leaving 300 W headroom for the NVME and RAM. I would also guess that there won't be many workloads that are using all GPUs and CPU simultaneously, but I can limit the V100s to 200 W if necessary.
- The RDIMMs are confusing me greatly. The memory QVL (https://www.asrockrack.com/general/productdetail.asp?Model=SIENAD8-2L2T#Memory) has three recommended modules, only one of which I can find anywhere. What are the chances I can get by with a couple of 64 GB DDR5 4800 MHz RDIMMs that I pick up from Amazon or the like?
Thank you in advance for any help / guidance / advice!
Component | Name | How many? |
---|---|---|
CPU | AMD EPYC SIENA 8534p 64 Core | 1 |
Motherboard | Asrock Rack SIENAD8-2L2T | 1 |
RAM | NEMIX RAM 256GB (4X64GB) DDR5 4800MHZ ("compatible with MTC40F2046S1RC48BA1") | 1 |
Storage | Crucial P3 Plus 4TB | 2 |
Case | Phanteks Enthoo Pro 2 Server | 1 |
CPU cooler | ARCTIC Freezer 4U-M | 1 |
PSU | Corsair AX1600i | 1 |
GPU | Nvidia Tesla V100 32GB PCIe | 4 |
Case fans | Phanteks T30-120 | 9 |
GPU fans | ARCTIC S8038-7K | 4 |
Fan hub | ARCTIC PWM Case Fan Hub | 2 |
Subreddit
Post Details
- Posted
- 5 months ago
- Reddit URL
- View post on reddit.com
- External URL
- reddit.com/r/homelab/com...