Workstation build for animation (Second build ready).
shambleresp_eb83ebd029
Posts: 45
Hi, want to build a new PC for animation (windows 10, without monitor,mouse and keyboard, own those), my budget is around 4.000 $ and have any doubts, want to put as many graphic cards i can (own a titan x maxwell and will buy gtx 1080's), was reading about that here and there and can't decide the way to take, a motherboard, chipset and processor who support more than 4 cards? or perhaps 2 PC's with 2 graphic cards each one? because i think the iray-render performance drop for each graphic card i add, any advice will be greatly appreciated, ty.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
I will be a detailed but short as I possibly can. Gpus (graphics cards or video cards) use something known as "lanes". Lanes are provided by a processor. Mainstream intel cpus are 16-20 lanes. While higher end i7's and xeons are 40 lanes. If you have heard the term "PCI Express X16" that 16 is lanes. Any gpu running higher than pci express x8 3.0 does not gain much benefits through gaming and rendering. Therefore 32 lanes pci 3.0 would be what you need. This means mainstream i7's such as 6700k will NOT BE ENOUGH as it provides 20 lanes. That would mean gpu 1 = 8x, gpu 2 = 8x, gpu 3 = 4 x. 8+8+4 = 20. You would need a high end I7 or xeon. For I7's there are 4 series (4820k,4930k,4960x) 5 series (5820k, 5930k, 5960x) 6 series (6800k, 6850k, 6900k, and 6950x). The 4820k, 5820k, and 6800k are 28 lane cpus so that is 8 + 8 + 8 + 4 for gpus 1-4. That is still NOT ENOUGH. You would need at least 32 lanes but there are no 32 lane cpus. The others in those series I mentioned are 40 lane cpus and they will be more than enough. There is also motherboards with PLX chips that add the lanes. However, those boards are more expensive and there is "LATENCY" on those added lanes which still gets looped back through the cpu. In short motherboards with PLX are slower lanes. Some will tell you to get a 5820k or other 28 lane cpu and use a motherboard with PLX chip but I would NOT RECOMMEND IT. The 4 series of high end I7's are socket 2011 while the 5 and 6 series are socket 2011-v3. There is also the 3 series but if I am correct the 3 series supports 40 lanes to pci express 2.0. In short pci express x16 2.0 = pci express x8 3.0.
With that tid bit of information said I will now recommend you these parts.
1. Big case with much breathing room. Lookup Thermaltake CoreX9
2. Power supply of at least 1200 watts. (1500-1600 watt PSU's would cover nearly anything you throw at it in the realm of 4gpus that is.
3. Minimum 16gb's ram. (This will vary depeding on peoples need but a good rule of thumb is twice the ram of your vram. So with titax x maxwell being 12gb vram you use 24gb ram in system minium.
4. SSD's of course. Your choice on size but I recommend 256gb at minimum.
5. High end I7 with 40 lanes (5 or 6 series)
6. Motherboard that has 4 double spaced pci express x16 3.0 slots. (If not dual space slots you are either custom water cooling or rigging the case with hanging gpus.
7. Graphics cards (READ THIS SLOWLY). The 900 series of nvidia gpus (970,980,980ti, Titan X maxewll etc) have cuda support. The 10 series may not have it yet for iray. Therefore they arent much faster than the 9 series if at all. Many gpu renders need the cuda support still for 10 series. Redshift, Octane, Furryball, Iray, etc. Also get gpus with blower style fans. Not the 2-3 fan gpus. Blower style gpus breath better when they are squished next to eachother and push the hot air outside the case.
If you are rendering animations I would not recommend Iray but would recommend REDSHIFT3D instead. It is a BIASED renderer were Iray, Ocatane, Furryball, etc are UNBIASED RENDERERS. You much do your research and find out what suits you best and what BIASED VS UNBIASED means. Do not be thrown off on what people way that UNBIASED produces better results or vice versa. DO YOUR RESEARCH.
Thank you very much HolyEternia, alot and good information, knew many of those aspects but now it's all more clear, the 10 series drivers for daz-iray are in beta http://www.daz3d.com/forums/discussion/95436/daz-studio-pro-beta-version-4-9-3-117-release-candidate-2-updated/p1 (last pages) i'll continue my research.
You can check out the website of workstation build companies like BOXX or Puget Systems, etc... You can see what kind of configurations they build to. BOXX has a nice website to easily see the builds they do. I believe the Apexx 4 is the 4 GPU chassis.
if could animate on a crappy Dell inspiron laptop with 2GB RAM any system is possible, it just comes down to how fancy you want it to be, in my case I could not run D|S 3 using 3Delight which was what existed at the time so used Poser and Carrara and a lot of compositing (both do animated textures and backdrops so relied heavily on that)
I have an i7 with 16GB RAM and a 980ti now but must say having crap hardware never stopped me before, just used more workarounds and compromises.
Animation need not be photoreal after all.
These days you have stuff like Hitfilm and countless other video compositors to do the job even better.
You can render everything to sequenced png and layer it in that, figures, props, backgrounds etc the paid version even imports alembic mesh (I only have the free express).
All the pros compsosite video, look at any CGI youtube blog on major films.
So no need to load all your scene on your graphics card, thats just nuts.
Thanks fredmusic and ToeJam for your advice. Well i have a first approach, let me know your opinion please:
1/ CPU: Intel Core i7-5960X 3.0GHz 8-Core Processor.
)
2/ CPU COOLER: Corsair H110i GT 113.0 CFM Liquid CPU Cooler (or just aircooler?)
3/ MOBO: ASRock X99 WS-E EATX LGA2011-3 Motherboard.
4/ MEMORY: Corsair Vengeance LPX 32GB (4 x 8GB) DDR4-2666 Memory. ( or 64GB now or
later)
5/ STORAGE: Samsung 850 Pro Series 512GB 2.5" Solid State Drive.
STORAGE 2: Seagate Barracuda 2TB 3.5" 7200RPM Internal Hard Drive. (Have another
external 2TB hard disk for backups.)
6/ VIDEO CARD-1: Asus GeForce GTX Titan X 12GB Maxwell Video Card (Own)
VIDEO CARD-2: Geforce GTX 1080. (will buy with this build.)
VIDEO CARD-3: Geforce GTX 1080. (to buy later.)
VIDEO CARD-4: Geforce GTX 1080. (to buy much later
VIDEO CARD-?: is possible to add any more ghaphic card? (NO ONE IN SLI)
7/ POWER SUPPLY: EVGA SuperNOVA T2 1600W 80+ Titanium Certified Fully-Modular ATX
Power Supply.
8/ OPTICAL DRIVE: Pioneer BDR-209DBK.
9/ CASE: Thermaltake Core X9 EATX Cube USB 3.0
10/ CASE FAN: Cooler Master R4-S4S-10AK-GP 60.9 CFM 140mm - 200mm Fan? (don't know if
put air or/and water coolers.)
11/ OPERATING SYSTEM: Microsoft Windows 10 Home OEM 64-bit. (Pro?)
Monitor, will use a 47" LED TV and a 50" 4K TV, and own the keyboard-s and mouse-s.
You might want to double check the case.
You're trying to fit an Extended ATX motherboard into a micro ATX case!
Fixed, a copy and paste issue
ty prixat.
A "nice to have" on the motherboard would be USB 3.1 and Thunderbolt.
How do your prices compare to the ASUS equivalent:
https://www.asus.com/us/Motherboards/X99E_WSUSB_31/specifications/
Ok, after alot of time researching i refined my build (i hope), any advice will be great:
1/ CPU's: 2 x Intel Xeon E5-2683v3 (used)
)
2/ CPU COOLER: 2 x Enermax Fit T.B. Silence
3/ MOBO: Supermicro X10DRX
4/ MEMORY: Kingston ValueRAM (ECC) 64GB 2133MHz DDR4 8x8GB.
5/ STORAGE: Samsung 850 Pro Series 512GB 2.5" Solid State Drive. (170 euros)
STORAGE 2: Seagate Barracuda 2TB 3.5" 7200RPM Internal Hard Drive. (71 euros) (Have another external 2TB hard disk for backups.)
6/ VIDEO CARD-1: Asus GeForce GTX Titan X 12GB Maxwell Video Card (Own)
VIDEO CARD-2: Geforce GTX 1070-1080. (will buy soon.)
VIDEO CARD-3 and 4: Geforce GTX 1070-1080. (to buy later.)
VIDEO CARD-5, 6..: Geforce GTX 1070-1080. (to buy much later
7/ POWER SUPPLY: Super flower Leadex 80 Plus Platinum 2000W Modular. (400 euros)
8/ OPTICAL DRIVE: Asus DRW-24F1MT DVD 24X M-DISC (16 euros)
9/ CASE: to make on my own to apply risers etc.
10/ OPERATING SYSTEM: Microsoft Windows 10 Pro OEM 64-bit.
Those xeons are for low power, low heat situations and have very low frequencies to achieve that.
You'd be better of with a faster quad core, the parts of DS that are single threaded would appreciate it.
Ty prixat, updated the micro, mobo, ram etc, better now?
The E5-2658v3 does not have 'Supplemental SSE3'. It's absence seems, possibly, maybe to be the cause of the current Iray beta crashing with older AMD processors.
I believe there are clear limits to what we should expect from the hardware most of us can afford. Video rendering for the common man is only beginning.
One thing that seems conspicuously missing is a video card that can accept user-installed RAM, like a motherboard. A bigger mystery is, why video rendering software doesn't save only the difference between frames, like an animated GIF does. It makes a smaller, faster file, and works well. What seems most obvious is that real-time rendering is the grail. Interactive Video, aka "gaming," couldn't exist without it. It's not clear to me why there is any question about it.
All these things are existing technology that, in some cases, has been around for years. When we have them, 3D animation will be more accessible than it is. Having recently installed a 980ti, I don't think even four of them would do an impressive job rendering a 10 minute movie in 1080p. What I'm getting at is, when it comes to building a powerful box for making 3D movies in the home office, there is a point of diminishing returns, and it may be closer than we realize. I say this especially to those whose hardware is not the latest and greatest.
..................
That said, I would build a 2-box system, because no matter what kind of hardware you have, rendering will bring it to a crawl. Install all the content on a NAS box, then use a utility to sync it to the other two boxes. Render on the box that is the beast. When it's time to render, you still have your other box to play on.
Have you considered going the pci-e expansion route? For example the Amfeltec GPU-Oriented PCIe Expansion Cluster http://amfeltec.com/products/gpu-oriented-cluster/ ?
I'm currently researching this route as I'm at my limit of two GTX 970s in my Z68 computer and would rather put more money into additonal GPUs without having to build a whole new rig.
Ty again Prixat for the advice.
Rottenham, an interesting point of view, perhaps i can use/modify my old PC and use it how a NAS. Also i'm reconsidering one of my first ideas (in the first post) and make 2 easier PC's and combine to do the animation/renders.
garyh.pub, that's impressive! i'm on it.
Also i was thinking about the motherboards with only PCIE 8x 3.0 slots, anyone knows if the PCIE 16x 3.0 has a better quality look in DAZ STUDIO?
Here's a good article by an Amfeltec customer, Elastic Pictures, on their search for render node expansion options. Some, like the Cubix solution is over $3.5K, and that's just the box and interface.
They chose the Amfeltec single cluster and at least in Octane are achieving benchmark numbers within 4% or so of the average OctaneBench results for the sum of the cards in their system - that's two 780's in the computer and 2 x 980ti's, 1 Titan X, and 1 Titan Z on the Amfeltec.
I just got a quote back from Amfeltec, their GPU-Oriented PCIe Expansion Cluster is $453.27 USD.
http://elasticpictures.com/1662/octane-render-render-node-expansion/
Nothing to do with quality. Possibly an insignificant effect on speed.
You can see from that Amfeltec review, they are running that cluster through a single 4x slot, (and they have connections to run another 3 clusters through the same slot!)
Looks expensive...........and still a movie will take days to render out in HD 30 frames a second....lets say 30 minutes per frame (if you don't use glass or caustics since they clog up the render engine quite dramatically. do the math 48 frames a day or about a second and a half, A freind of mine has an Apple workstation but these days he renders using a cloud service......cost 30 euro's and in a few minutes result. If the client has a render budget, no problems.....time is money and waiting a week for a render you don't like......not a good idea when deadlines loom on the horizon, 30 euro's is not even the power cost of a render on the workstation.....not to mention investment in the beast.....Apples are expensive......high end Apples are crazy expensive (about 2 as much a you envisioned workstation!
Think about it.
Greets
you can render a lot faster than 30 secs a frame depending on settings
for animation photoreal is really unneeded and often a bit creepy uncanny valley
Very valid point but often ignored overlooked.
Thank you guys, just remember Renderosity is giving a present every day, it's called "Countdown to Christmas!" just click in the calendar, started yesterday...
Well, changed the RAM to 8x8GB because in a dual quad system better fill all four channels, in other words put 4 RAM sticks for each micro to avoid a bottleneck, motherboard or computer manufacturer should be able to tell which slots to fill for Bank 1) Usually A1,A2,A3,A4 for CPU 0 and B1,B2,B3,B4 for CPU 1.
Also changed Windows 10 Home to Windows 10 Pro, seems only Pro can manage 2 chips.
Hey guys, next step, my concern now is about the PSU, with the posibility to install a maximum of 10 GTX 1070-1080 or Titan X graphic cards that need a x6 and x8 PCIE pin connectors each one, can a PSU provide those 20 pin connectors? (x6+x8) or i need to install more than one PSU?