Building my own little render farm on the cheap
Jonstark
Posts: 2,738
Last night I disassembled my oldest laptop and changed the cpu from an i5 (4 rendering cores) to an i7 (8 rendering cores). Now I've got 2 laptops and 1 desktop, all of which have i7's (though they are all different generations, 2nd, 4th, and 6th respectively, this still means 8 threads of rendering each, and when network rendering I've got 24 cores of rendering now).
I picked up the i7 cpu used from ebay for just over $60, pretty inexpensive, but after doing that I started poking around wondering what kind of prices there were for full pre-built desktops that had i7 cpus. Wow, was I surprised; I can clearly remember when you had to shell out huge amounts of money to get a computer that came with an i7. That's still true if you're going to the local Best Buy to pick up the newest off-the-shelf brands, but scanning through ebay for used computers I was struck by the fact that it's not very expensive at all to pick up an i7 pre-built.
I suspect the reason has much less to with regular consumers selling their used devices as they upgrade, and more to do with large company's who have upgraded their employee's fleet of workstations to the latest/greatest and are selling off the earlier generation stuff. I found I could pick up a 2nd generation i7 desktop (in a small form factor compact size btw) for around $200, and a 3rd generation i7 for around $250 (I can see there are 1st generation i7's desktops for around $160). Are these earlier generations of i7's obsolete somehow for my purposes? No way! They all have the potential to add 8 more render cores to any network rendering I do in Carrara, and on top of that these are desktop i7's so they outperform my laptop cpu's even if the laptops are running more recent generations of i7's.
Because I'm greedy, and 24 cores just isn't enough (I want to see the whole render screen filled with cores 
) I've decided that at these prices I can add some more computers to my network and make my own little render farm. I already went ahead and bought a 3rd generation i7 desktop tower (i7 3770), which should be coming to me later this week, and then I'll be at 32 cores once I've got it all set up. But already my inner greedy render pig wants more; what stop at a mere 32 cores when it would be so easy to have 40? I think I'll pick up one of the 2nd generation i7 (i7 2600) very soon, and maybe even get one of the 1st generation i7 systems, just to round it out so I have one of each (and can compare/contrast the render speed differences in real time).
On top of that, I've already got the bug to build my own desktop sometime in the coming year. I'm not a tech guy, but thanks to many instructional videos on youtube that are freely available, I've been able to disassemble/reassemble both of my laptops, cleaning out all dust and re-applying thermal paste (so important to keep the cpu temperatures running low, and so easy to do), and now having a much better understanding of the components and how they work together, I'm eager to try to assemble a new desktop myself. Thinking of going with a motherboard that supports a socket for one of those big 6 core (hyperthreaded to 12 cores) i7's. More render cores! (my new motto)
But I was watching a review of the i7 6950X, which 10 Cores (!) hyper threaded to 20 cores (!!) but which is over $1700.00, just for the CPU all by itself, and as awesome as it would be to have that cpu, I can't help thinking "Wait a minute! For way less that $1700 I can pick up a bunch of older generation i7 desktops and put them all together in a network to have way more than 20 core rendering.
It just really struck me that putting together a personal renderfarm has become a lot more economically viable.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
I have also been looking t the goverment resell stite here in canada and I can get some older end of life servers for cheap as well as the desktops so I am always looking at that side as well they have older GB switches in 48 port configs which make it even better.
My original plan when I built my machine (8 core AMD Zambezi - no longer available new) was to eventually build a second and third render node running nothing but Carrara Rende Node and the stripped down OS. Basically node PC's with no other purpose than to render Carrara.
I can't remember where to find out how many cores we can run before needing to purchase GRID - do you know? Is it 20 separate render nodes?
I was looking around and found a site claiming that you can have up to 20 nodes natively in Carrara, but bear in mind that 'Nodes' isn't the same thing as 'Cores', I think in this context Nodes just means separate computer instances running Carrara. I don't know if there's a render core limitation or not (but I can attest that I've rendered with 24 cores on my mini-network myself now, and I know I've seen other screenshots from Carrara forum members rendering with as high as 36 cores (I think it was Stringtheory), but as much as I am keen on the idea of building my own mini render network, I have no plans to go as far as having 20 different computers running at once :) So I doubt I'll run into the node limitation.
Also another advantage to purchasing these inexpensive used i7 mini-towers is that they are so inexpensive that if it dies unexpectedly, no big deal. My last laptop was over a thousand dollars, and the desktop I just bought was (together with the Oculus rift which was my excuse for buying it) over 2 thousand dollars. If either of them suddenly dies, it'll hurt (but then again, since I'll have another i7 computer at the ready, I guess it will blunt the pain a little).
Also I see no reason that any of these devices should unexpectedly give up the ghost anyway. Heat is the mindkiller for computers (to misquote Dune) and now that I have the basic knowledge to disassemble/clean/re-assemble, plus add in whatever extra fans I might need, I think I'll get a very long lifespan out of them, and even if a specific part fails I know I can get a replacement part cheap on ebay and do the swap out myself. I'm a little amazed at how simple computers really are, now that I've done some studying up; I used to think of them as 'magic boxes' terrified ever to so much as loosen a screw on the chassis for fear that they would instantly die. But then I stayed one night at a Holiday Inn Express, and now I feel confident enough to do surgery :) At least, computer surgery.
Also I think I kind of had the idea in the back of my mind that a 2nd generation i7 must be twice as good as a 1st generation i7, and gosh if I don't have the latest 6th generation i7 then it must be a waste of time, but really it doesn't work that way, the later generations are of course better/faster, but not by so very much that it makes the earlier generations obsolete. It reminds me of that 3 stooges skit where they find a wallet and they're counting all the $100 bills: "$100! $200! $300! $400! Oh, a $50, how'd that get in there..." and they ball up the $50 and throw it away (before exchanging glances with each other and diving after it). :) no reason in the world to pass over a perfectly good i7 that can add render cores of goodness to your network. :)
Quick correction, I found the site where I saw that documentation, and it claims the natural max is is 10 Nodes and 20 CPU's for Carrara (Grid can do 50 Nodes and 100 CPU's). CPU does not equal render cores however, as I'm able to network render with 24 cores at this very moment and I don't have Grid, so the CPU is counted by the actual number of CPU chips, and if you have 8 cores from a single CPU it doesn't look like there's any restriction on that. I guess if someone was really serious about building a Carrara renderfarm they could get Grid and put together 50 different servers with dual CPU motherboards on each server.Theoretically if they put an i7-6950X in each CPU slot, that would be 20 cores per CPU, or 40 cores per server x 50 = 2000 cores to render with.
This way you don't even have to buy a heating for your home!
Just what I was thinking! I have just one i7 tower PC that I built earlier this year. Between the cpu, graphics card and monitor, it keeps my little office toasty warm, to the extent that I wouldn't contemplate a second PC in the same room (I have other rooms need heating, lol). Just don't underestimate the amount of heat these things kick out!
That's just gorgeous, isn't it? Oh so Sweet!!!
I'm already tickled pink at how blazingly fast my octa-core can render. I'll plead to Rosie and get her to let me eventually add some more core to my system! Oh this is such good news!
Even if I ever ended up having to buy Grid... it's a cheap way to add that many more freaking cpus!
Bravo, Jonstark! I'm very happy for you, and definitely want to join the club!
BTW, how easy is it to network them together. I've never done that. I'd love to be able to add my octa-core to aid in my laptop. Is that possible? My laptop is Win10 64 bit, my Octa-core is Win7 64 bit and I can hard-wire the LAN, but I just don't know how to actually create the network. Also... could I do it via wireless router instead of hard-wire? That would be more ultimate for me.
That's cool, Jonstark... you will be toasty warm this winter!
Not long ago here in Bristol, Yellow Dog started a render farm using individuals. You can earn about $8-10 a week just to leave your machine running!
Cloud and Crowdsourced 3D rendering for animators and artists
http://yellowdog.co/
Weird thing is that now that I've switched the i5 for an i7, the i7 actually runs cooler when rendering, though I have no idea why that would be. Weird.
Super easy to set up a network on Windows. I've got different versions of Windows on all my PCs too (Win 7, Win 8, Win 10, respectively). Just do a search for 'Homegroup' , use the little tool that will come up to set up a new Homegroup (name it, select what can be shared, set a common password for the homegroup that's the same on each computer that you do this to) and you're done. The computers will be able to 'see' each other through the homegroup at that point, and you'll see Carrara can see the other computers in the homegroup as 'available' and you can network render immediately (though do remember that you can only network render as a batch render).
Thanks! I enjoy Batch Queue anyways! ;)
So I just add that new homegroup on each computer and go, eh? Sweet!
Hey Johnstark!
Sounds like your having fun. Another thing to look at on ebay for your render farm would be zeon based workstaions, or servers. You can get a dual processor (hex core) used server for some excellent prices if you won't be using it for Octane. There are also some great deals on Lenovo workstations (Just search for lenovo d20 to find the dual processor units).
Enjoy your render farm!!!!
Thanks for the tip! I know almost nothing about the Xeon stuff, and what generation corresponds to which i7 generation, but a quick glance is showing me 6 core (hyper threaded for 12 render cores) servers at under $200.00, some as low as $150, while I can also see the lenovo d20's you mentioned going for as low as $300. Sounds like a steal to get 12 more render cores for such a low price. I just don't know if servers take something different to setup and use than regular desktops do, or whether I can just expect to use Windows 7 (or whatever) and install Carrara normally. I wouldn't be using these for any other purpose except rendering Carrara, certainly not gaming or internet browsing or streaming video, and I assume a server can do rendering just as well as a regular consumer-grade desktop, but I don't want to make a mistake.
As an aside, I don't want to give the impression I'm abandoning Octane. My new gtx 1070 freaking flies when it comes to Octane rendering, and I've never been happier, but I love me some Carrara native rendering too (plus especially if I need to do a Howie or TangoAlpha level scene with lots of Carrara trees I can't do that in Octane). Just looking to round out all my options :)
Oh and I know you're big into rendering on laptops too, like I am. I was shocked by how easy it was to take apart and clean the dust from my laptop fan, and how the idle and rendering temp just dropped nice and low after doing so. I think you're technically a lot more proficient than I am, so you probably already knew about all this stuff, but it's a giddy feeling of accomplishment to know I can keep my laptops in good working order :)
Me too... same predicament.
BTW, that's 24 more cores - six cores HT to 12, times two! At least for the d20 (I think... I didn't look - but we're talking dual cpu servers here!)
Our pal, Garstor, whom has now switched entirely to LW :( builds such machines. He also uses (provided by his work) the really expensive server-class Windows OS, but I don't think that's a requirement - and also, these used machines already have the OS...
I have a feeling that our Carrara Render Nodes would just install as normal - but it's certainly worth looking into.
Ahh... looks like the d20 comes with 2 dual core or 2 quad core cpus and Vista Business (High-End) version of Windows, which would likely run Render Nodes just fine... but that's only a guess
https://support.lenovo.com/us/en/documents/pd005619
I need to research more info on the motherboards, to see if they are dual-socket (meaning they can take 2 different Xeon cpus) or not, the ones listed on ebay I saw only showed as coming with 1 Xeon (so 6 core to get to 12 rendercores) but if the motherboards are dual socket and can potentially take another Xeon, then that's certainly worthwhile indeed, as I can just purchase another Xeon cpu from ebay (or wherever) and slot it in to increase it to 24 rendering cores (whoa, I can just picture one box adding 24 cores!
) Would certainly be worth the expense of purchasing another Xeon cpu on top of the server itself, assuming that it's as simple as adding another cpu to the motherboard and doesn't require some complex programming to make it all work together which might be beyond my skillset...
Just googled for a picture of what a d20 lenovo thinkstation motherboard looks like, and it sure looks like it's got 2 cpu slots on it, so that makes me very happy to see. Now to chedk if the less expensive Dell T3500 servers are dual socket or not...
Ah no, looks like only 1 cpu slot on the T3500, which probably explains why they are less expensive...
On a server you can run windows 7 or 10 if you want or you can purchase windows Server 2012 R2 for it.
Thanks Chickenman. As long as windows 7 will work, that's easy peasy for me, I'll just buy a win 7 coa and install. But does it have to have any special setup to use dual cpu's? I'm thinking of getting one of these that has dual xeon cpu motherboards to double the number of render threads.
In fact, I might just build myself a server from the component parts on ebay. I'm seeking xeon e5-2650 which are 8 core (hyper threaded to 16) for just around $40 bucks a pop over there, I could pick up 2 of them for $80 and would be looking at 32 threads of rendering right there, if I can put them on a dual socket motherboard...
Hey... get me one of those too... I'll pay you back... um... some day! ;)
Sure, at $40 a pop I can give 'em away like Christmas cards
Well, maybe not quite... And also I'm terrible at giving away Christmas cards, so that's really bad analogy.
Looks like it would be the dual socket motherboard that would cost more, not the CPUs themselves so much, I'm seeing the lowest price motherboard that might work is still $200. That starts to hurt, and I'd still have to have... what? A case, fans, power supply, heatsinks, hard drive, ram. I don't think I'd need a graphics card though since I'd only be using it as a render core, and I'm not sure I'd even need a dvd drive as I think I can just install Windows from a thumb drive... maybe this might be doable after all...
Can anyone who knows who to put together servers let me know if this is something that a regular non-techie idiot like me can do? Or would I run into tons of problems if I took this approach?
The bigest thing to watch out for is that the the processor (zeon) supports hyperthreading. You don't see a lot of the older non-hyperthreaded versions any more, but there are a few (just google the cpu number and go to intels web site to see the specs). For raw numbers crunching (rendering), xeons are quite good. Like i7's, the newer versions give the best performance. Probably due to their intended use (24x7x365), it's been my experience that xeon motherboards in general tend to last longer than desktop boards (was in charge of an IT department for 12 years, and never had a xeon board go bad, but would often have a series of desktops where the motherboard would start failing right at 3 years).
With servers, because they probably were on 27x7x365, you may have to replace a fan or two, a disk drive, or power supply (fan issues). The workstations probably didn't get as much around the clock use. Servers are pretty much bare bones systems compared to desktops, and often are desogned to be mounted in racks and not sit on the floor - so this is another thing to consider (they do come in tower cases too, which may be preferable). Servers are designed to be a bit more "fault tolereant" so they usually usually have redundant power supplies and hot swapable drives (when running a raid array, and one drive dies, you pull the bad one, put the new one in, and it's all automatically rebuit and the server doesn't "go down").
For your purposes, a workstation or a tower server would probably work better. I'd definitely go with a dual processor unit (you will need Win 7 pro to support 2 processors). A hex core will give you 24 threads, where a quad core will give you 16. Many applications don't scale well past 6 core and 12 threads per processor, so even though an 8 or ten core processor sounds awesome with 16 or 20 threads, you may not see much improvement over just 12 threads (unless of course you are running an application specifically designed for massive parrallel processing, which Carrara probably isn't, or if you are running multiple applications simultaneously).
One big advantage of the xeons over the older i7's is the amount of RAM supported by the board. Most of the older i7's max out at 32Gb, with dual processors, the Lenovo D20 maxes out at 192Gb. Of course that seems like a rediculous amount for Carrara, but 48 or 64 Gb could really give you a bit of flexibility (you could render 2 large Carrara scenes at the same time on a single computer). With used Dell workstations, you'll want to start off with dual processors because the second one requires an add-on "cage".
Here is a really good deal on a Lenovo ThinkStation D20 Xeon 2x X5560 2.8ghz Quad Core 32gb 2x 1tb FX4800 Win7, for $395. I'm drooling over a Lenovo with 128Gb of RAM for work I'm doing with Agisoft PhotoScanner, but my wife doesn't want the noise (dual processor systems can be a bit loud
Anyhoooo, have fun setting up your render farm!
I always find it kind of funny how many people seem obsessed with making their pc fans as silent as possible. I'm the opposite; before getting this new desktop I always had a white noise generator on in my room all the time, as I find it soothing and it helps me sleep, now loading up my desktop with every fan slot it could take, and also with 3 fan coolers under each laptop plus a vacuum fan on the vent of each laptop, I have to say even though I wasn't going for quiet, they aren't very loud and secondly I don't mind the noise at all. So the thought of fans in a server don't bother me.
However while I'm not a tech guy, I do happen to work for Cisco (in a non-tech role) and I often find myself talking to guys over the phone who are in the midst of a server room, which comes across the phone line as nothing less than a roar they have to practically yell over. I'm guessing that's probably hundreds and hundreds of machines though to get such a different noise volume
Thank you very much for the tip. I was looking that server you linked over and it looks like it comes with 2 xeon quad cores, which would mean 16 rendering cores, but if I'm right about the socket they go in, looks like down the line I could swap them for 2 6 core xeons instead, which would bring me up to 24 rendering cores. Sadly I couldn't find an 8 core xeon that would fit that socket, so I don't think I could get up to 32 cores and live the super high life lol, but this would be a great place to start, and it comes prebuilt with parts I know fit together and work already. Does setting up one of these things require any special monkeying with the Bios? Or is it as simple as turn it on, load up Windows 7, join the homegroup network, install and start Carrara render node and then render away?
Ironically I can already tell with this new obsession I'm going to end up spending way more time and energy putting together my ideal render network and then realize I haven't put together any scenes that need network rendering yet
Sounds fun!
I remember the first time I connected a 2nd machine and how much faster things went; I really wish that CMD-R would engae the nodes, but that's a minor thing.
Purely Mac here, so my choices are fewer. I actually get faster renders off of my iMac with 4 cores than my Mac Pro with 6, and the Mac min pulls in last. The mini gets hot after just a few minutes and whirs like a UFO about to take flight so I only put it into the mix when I have a few shorter things to do that I want to get done fast. The mini I have has a quad core, but the new ones only have dual, so I don't know how that would impact Carrara. I am considering getting a cheap i5 machine just to see how it fares. . . If it does OK, then I could add another and another and another and .. . where's Santa Claus when you need him?
About silence, I cannot tell you how much that means to me. I record voice and in the past had to run a mic cable from the computer to another room to avoid fan noise, even with low strain on the CPU. With the new machines, everything is within a meter of where I sit and you can barely tell any of the machines are on, even when they are at 100% CPU load. Less and they are silent. The Pro does produce a bit of heat, though, sothe room AC kicks in. Much less of a problem in winter!
Because they can cause hearing loss if your fans are running loud over the years. Me, for example. All it takes is an ear infection to tip it over the edge and for you to wake up one morning with tinnitus and no directional hearing for the rest of your life ;)
Tinnitus is not fun. I've had it for years (not from IT work). Do whatever you can to protect your hearing now; once tinnitus starts, it is usually permanent.
I've had mild tinnitus since I was a teenager actually, it's part of why I prefer there to be some measure of white noise, as in absolute silence I start to get a 'feedback loop' (best way I know of to describe it) where the high pitch ringing seems to get louder and louder for me. I used to be the singer for a hard rock band, which is where I probably got it, but I'm not too worried about some soft humming of fans making it worse at this point, and most of the time I don't notice it anyway.
Opps, sorry about that
, a couple of weeks ago there were some 2x hex cores in that price range, but it looks like all the 2x hex cores now are starting the $430 range now.