Render Farm
2getit
Posts: 0
I would like to start building a render farm to speed up some animation renders, what is the best way or easiest way to do that in daz studio?
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
DS does not have network rendering capabilities...
Ok, just curious, what is the RIB export for when rendering using the default 3delight engine? I've noticed that you can network renders using RIB using third party engines, i think
Yes. But there are big drawbacks when trying to do animations, that way
One of the biggest...the free 3Delight stand alone is also not network capable...and converting a DS RIB to another Rman renderer isn't easy (too many shader conversions to make it easy). So the best you can do is a second machine...not really a 'farm'.
If you want to pay for the standalone 3Delight, then I suppose it would be practical.
In the render settings select "Render to": "Image Series" and assign different start and end frames to the various computers. Then rebuild the video in one of the several available programs (VirtualDub, Windows movie Player, etc.) One could conceivably keyframe the renders on a plane and then render that to video inside DS. :-P It would be slower than doing it the other way, though.
the ability to export .RIB's was very useful and still is for older systems or systems with limited RAM because it allowed you to render in an session that Daz Studio did not need to be running in and this freed up a considerable amount of RAM. If you had a 2 core system it was overall faster to render (by a few seconds anyway) but Daz3d does not own or partner with 3Delight, the 2 core limit is their decision and they don't appear show any sign of changing that anytime soon. Standalone 3Delight for network and use with more than 2 cores will cost $2,500 US for the license.
If you have Studio 3.1 you can use either Reality ($49-$69 US) or LuxRenderDS (free) to export to LuxRender (free) which will allow you to map a render farm to your host system provided all the nodes are running LuxConsole (free and bundled with LuxRender)
If you have Studio 4 you can use Reality, but not LuxRenderDS
I have used all three solutions, personally I prefer the Reality to LuxRender farm one over the other two, possibly because it was far simpler than the 3Delight solution and I got better results. Reality has a great support system, and my dealings with the developer have all been very positive and professional.
Here's an old post on the .RIB to external process, ancient stuff but it still applies for the most part.
http://forumarchive.daz3d.com/viewtopic.php?t=109277&postdays=0&postorder=asc&start=0
re:blue -- This is not quite right. As long as the shaders are RiSpec, they'll be compatible across Renderman Renderers. That is what RiSpec is about. When one exits into the non-RiSpec areas is when things get hairy.
Animations are doable. One just has to plan for them. I just completed a 120+ frame RIB'd animation containing Optitex Dynamics and other "interesting" items.
The commercial version of 3Delight has built-in network rendering (up to the number of licenses that have been purchased)
Kendall
re:blue -- This is not quite right. As long as the shaders are RiSpec, they'll be compatible across Renderman Renderers. That is what RiSpec is about. When one exits into the non-RiSpec areas is when things get hairy.
Aqsis doesn't do 'auto recompile'...so all the DS defaults, Omni and others that are in sdl only form aren't usable, without the source. Pixie hasn't been updated in a long time, so it's feature set is limited and many of the more advanced/recent shaders won't work, because the features aren't available in Pixie. Prman may...but that is definitely a more expensive option. As to others...mileage may vary.
Aqsis doesn't do 'auto recompile'...so all the DS defaults, Omni and others that are in sdl only form aren't usable, without the source. Pixie hasn't been updated in a long time, so it's feature set is limited and many of the more advanced/recent shaders won't work, because the features aren't available in Pixie. Prman may...but that is definitely a more expensive option. As to others...mileage may vary.
Compilation is non-RiSpec. Each vendor is open to compile to whatever form they wish. Just as the C++ standard is for the source only. Micro$oft's compilation is not the same as the compilation from g++ . However, both are C++. Renderman is the same, only the Source code is RiSpec.
Kendall
Right...and the ones that don't ship with source code, can't be compiled. Which pretty much means the DS defaults and the Omni (if they are supposed to have source, now, I can't find it).
This is an issue with the shader vendors, not Renderman, RiSpec, or RIB. The point is that there ARE shaders out there that don't ship compiled or encrypted and those will (mostly) work without problems. Which is why I said "not quite right." There are those instances where a commercial concern has limited the access.
My point was that porting the RIB is not a problem. The specific shaders used may be. The solution is to not use shaders that are restricted, or to use shaders with analogues in the target environment.
Kendall
Porting the RIB and even dealing with the shaders are not the biggest problems. The biggest one is to make DS to export multiple frames to RIB in a batch. When I select "render to image sequence" or "render to movie" and then "render to RIB", it puts the first frame to the RIB, then clears it and starts with writing another frame from a scratch, ending with just the final frame in the RIB!
The RIB contains only WorldBegin, no FrameBegin section, so it's made like separate RIB per frame but written into the same filename. I don't know how to overcome this bug - I'd be happy with either one big RIB with each frame in separate FrameBegin - FrameEnd, or with separate RIB files per frame, but I can't accept exporting every single frame manually! Please help!
Basically, what you are looking for is either the RIB to be written 'correctly' with FrameBegin/FrameEnd or the file name to be incremented after each RIB is written...
With the way DS is set up, it would probably be easier to come up with incremented RIBs...
Casual has a script for doing just that with images...I'm not sure if it can be adapted to render incremented RIBs, but...
https://sites.google.com/site/mcasualsdazscripts/mcjlistrender-for-ds-1-2-3-4
reality 2 will allow you to create a render farm on a network or over the internet. I am using it with Daz 4.5 and luxrender.
I know it's a bit old thread but I wish to share my method of doing render farm on DAZ Studio.
You can split # of frames across to computers with DAZ Studio installed (you'll need to share your My Library folder for other computers to access). Set up a ratio of # of frames based on their benchmark. That way all computers can complete the entire renders at the same time. If a computer is twice the fast, give twice the amount of frames to it.
In my case, my three computers are based on the same benchmark so I just give each equal # of frames.. I was able to complete the animation render in 1/3 of time.
Any thought? :o)
Or check your EULA to see if you can install the contents to rest of your computers for speedy loading & rendering.
In a render farm for LuxRender, no limit to the number of processor cores?
no limit to them other than the licensing cost.
DAZ Studio does not export multiple RIB files for animation... It exports a single RIB file. If this is somehow an animated RIB file (I don't think there is such a thing) I can't figure it out. So until DAZ fixes the RIB export, you have to render to RIB one frame at a time.
Don't forget that if you are working with animations you can still install DazStudio on a bunch of machines and have each machine render a part of the animation. That will give you the benefit of throwing more cores at a single project. So let us say you are doing a 10 minute animation at 30 frames per second. And let us say that each frame takes 3 minutes to render (not an unreasonable assumption.) That means 18000 frames at 3 minutes a frame for 900 hours of rendering.
Install DazStudio on 2 machines and the same animation takes 450 hours of real time. 4 machines makes it 225 hours or real time.
so the secret to doing it this way is to render to individual frames and then assemble the finished animation using something like VideoMach and assigning each machine a reasonable number of frames to render each.. in this case that might be 5000 frames.
Ok, now onto the fun part. It can be a real pain to administer to all those machines.. here is my solution for the issues that I use with my renderfarm.
1. Have all the machines connected to the same switch using a gigabit connection. This helps to reduce problems when transferring large amounts of data.
2. Have your main runtime installed in a shared folder. This allows you to easily copy the whole thing onto each machine for each DazStudio to access.
3. Use some sort of remote software to allow you access to each machine (I use RealVNC) so that you can remote to each machine and check on it's status, install updates, etc.
4. Use a KVM switch for your farm machines so that you have direct access from a single keyboard/monitor/mouse.
5. Turn on remote start for each computer so you can use something like WOL to remotely start up the machine. Set your machine to power down when idle. This will allow you to save LOTS of money on electricity.
6. Have a networked drive also connected to the switch. Mount the drive on each computer and have them all put the individual frames into a folder. This will make it easier to work with later.
My renderfarm has 5 farm machines and 1 main machine. I normally use it with Vue 11 infitinte, Poser pro 2012, or Carrara.. but there is no reason why it couldn't work with Daz Studio provided I installed studio on each machine and copied my libraries.
One thing to remember about renderfarms, however, is that your saving real time and not money. It costs me nearly $4 per day to run my farm. That means in 10 days my farm is running I increase my power bill by $40. I hope this helps!
Boojum the brown bunny
boojumbunn, that doesn't help those of us with only one machine and the need to render animations without locking up our equipment for weeks... :(
UMmmm.. that is correct. I thought the discussion was about building a renderfarm. By definition you sort of need multiple machines. For that matter, when I'm fully using my renderfarm I'm locking up 6 machines with my render. :)
If you have a single machine and don't want to lock it up rendering an animation then the only real solution is to hire a renderfarm with a 3rd party, but they tend to be more expensive than I prefer. Also, it can be a pain paying someone to render it for you only to find out you missed something in your files selection or they don't have a plugin that you do.
But yes, my advice is for if you want to build your own renderfarm.
Boojum the brown bunny
The nice thing about DS is that when rendering to RIB it'll gather all the files in a single folder so all I'd have to do is zip up that folder and fire them off... the problem is, it won't render sequential RIB files for an animation :(
Your right, it's about building a render farm... I'd be a client rather than a farmer :)
Yeah... I see what you mean. I just gave it a try. I wonder if there is a way using their scripting language that would create a render with the current settings, step the frame forward, do a new render to RIB with an updated name, etc? It wouldn't be the same as a multiframe RIB file, but could be a way around the limitation.
But your right, really they need to support multiframe RIB files which would allow you to put it all out in one folder, zip it up, and send it off.
:) For me, I'm a farmer and proud of it. I like being able to play with everything on my own. I set up my renders on my laptop, hook my laptop into my home network, and let 5 other computers help with my animations. So I've learned how to take care of my farm to make it easy to do whenever I want.
Boojum the brown bunny
As far as I can tell, the frame by frame render method is the only way to do it... exceedingly tedious, to be sure, especially if you're doing 10+ seconds... 300+ renders... then you have to type in all the files with their paths renderdl -id file1_collected/file1.rib file2_collected/file2.rib file3_collected/file3.rib 300 times... :(
Well, I know that Maya can output a multiframed RIB... allowing for animation inside a single folder. With any luck DazStudio will be working on that as well. After all, they already have the output for a single frame. Multiframe would be a good next step.
Boojum the brown bunny
Question, is there a command that will render sequential renders without having to type in every single rib filename?
The following test sequence was done from DS4.5 using RIB's output in sequence. The RIB's were fed to Linux boxes running 3Delight and rendered to a shared storage area.
http://youtu.be/gs6tqk1SD8A
BTW, the towel being worn is an upcoming Dynamic Product by Thespisis. This test was part of a test for the towel's functionality as well as to help layout part of a larger animation.
EDIT2: The original sequence lasts a little over 2 minutes.
Kendall
how did you get DS to export sequenced RIB files?
Can you describe me your computer configurations? $4 a day.. running at full load for 24 hours... I'm not sure if that's spiking or common in comparison.. I use Mac Mini's for render farming. Each unit will generate around 60 watts at full load.
I dont' have the stats on the power supplies at my fingertips. I'd have to regenerate them. They used to be in the Carrara forums before the forums were changed. :)
But they are 5 PC desktops that I put into rackmounts. I got how much power they used by using a Kill-A-Watts and then calculating based off of my PG&E power bill here in california. I'll see if I can get the stats for them next weekend when I have some time. But the $4 per day is for all 5 computers running 24 hours.
I just ran the calculations for your mac mini's and if they draw 60 watt's each then five of them would cost me $1.80/day to run. What CPU do you have in your mac mini's? So 10 days of rendering would cost me $18, a little less then half what the PC's I'm using now would use.
Boojum the brown bunny