Using Amazon AWS to render?

callmepaul

Hi! 

I am super new to DAZ and I am already hitting a wall with the render times. Yesterday I tried a render in my local machine and after 6 hours and lots of noises it was stuck at 4%. 

I am wondering if there is anyway I can use an Amazon EC2 GPU instances to remotely render my local DAZ files? I tried installing DAZ in an EC2 instance but running DAZ using RDP has a very bad performance. I can't drag things around smoothly and many other issues. But if I could use the 'power' of those machines it would be awesome.

Does anyone has any experience on this? Else, can anyone recommend any onlise service I could use to render my scenes?

Thanks a lot!

SimonJM

I suspect the answer would be no, as I thik you'd need to also upload the underlying geometry of the objects which is not permitted under the EULA.

callmepaul

SimonJM said:

I suspect the answer would be no, as I thik you'd need to also upload the underlying geometry of the objects which is not permitted under the EULA.

 

Thanks, so there is no other solution to render other than local resources? That is very limiting, isn't it?

SimonJM

I'm just a user, so my answer is not defintive ... the issue is the geometry, if that is copied to another system that is not directly yours there are EULA problems.  If it an be shown that the geometry is 'not recoverable' at teh far end you may have some wriggle-room.

callmepaul

thanks, technically I would own the EC2 instance during its life, so maybe that would not break the EULA. But still I am not sure how to convert my EC2 instance into a DAZ server for remote rendering, or even if that is possible.

Thnks again

mindsong

Mechanically, using a rendering engine that's more open/portable/available like luxrender to actually execute the remote renders seems more plausible to me than trying to get 3DL or Iray renderers lined up on arbitrary remote hardware services like AWS (considering both setup compatibility and licensing issues). Taking the luxrender route implies using DS/Poser/Carrara to generate luxrender rendering data that can be 'sent' to the chosen luxrender-farm (via luxus/reality/luxus-core/etc.).

FWIW, @callmepaul, I wonder if your assertion that you 'own' the remote renderers for the duration of their use would hold up legally. I think it might, especially as this geometry duplication is merely a multiplier of the already sanctioned processing and still clearly consistent with the intended end-user/purchaser benefit. This, as opposed to the kind duplication/distribution involved with using these mesh products in legitmate game publishing or simply illegal pirating distribution - the main difference being in the second case, *other* users are in control and possessing the (rightfully) copyrighted mesh information, rather than a person running local and/or remote renders for his/her own use - generating a non-mesh related result (one or many images) - as was clearly intended in the sales model.

To wit, I think the argument that a render being done down-the-hall differs in any notable manner from those that would happen across the net, so long as the information involved is completely non-public and under the purchaser's control for the duration of the processing, would NOT hold up if challenged. In deference to the true copyright holders, I also don't think such a judgement would have a notable impact on either party's (vendor/user) financial situation either (damages). Who would be hurt if an animation using, say, 20 copyrighted digital meshes were completed on one machine in 30 days, vs 30 machines in 1 day? I don't see the damages that would validate a lawsuit, so long as the data were controlled, transient, and never publicly available. (But I'm no lawyer..., I periodically suffer that rare malady called common sense, which probably has no place in such conversations...). At best, I could see needing 30 legit copies of a product if the EULA differs from the 'like-a-book' usage model (many copies are OK, but only one active use at any moment), But what if the user has a single free version (e.g. G3F that comes with DS) of a mesh. Do they have to re-buy/register 30 free copies of DS to use it that way, etc.? And... does this licensing constraint hold up legally, even if most people naturally honor/respect/follow it? Interesting questions in interesting times.

To be absolutely clear, I completely understand, follow, support, and respect the copyright and terms of all content producers and distributors (my legit and way-too-large DAZ product library speaks to this).

As this particular form of rendering duplication is both controlled (kept completely within the user's operating domain) and is unambiguiously consistent with the original intent, sales pitch, and advertised use of the original products (products licensed for end-user rendering use), does the act of adding efficiency for the purchaser, with no measurable expense to the copyright holder (that I can see) change the nature of the relationship enough to nullify the original contract? I wonder how the legal system would (or will) rule on such use.

I'm certain many conference-room conversations have been had at DAZ/Rendo/RDNA/etc. about this very topic with lawyers and CFOs involved, It's clear that without good copyright protection this industry would not exist/survive. I'm not sure 'punishing' legitimate customers for using these various available efficiencies will work to the industry's benefit - witness the whole Connect fiasco. I would think that more cool animations coming out of the customer-base would mean more users getting drawn into the business and more of them buying content.

The good news is that this is all interesting stuff that fortunately doesn't impact most of the non-animator market-place, so most folks probably don't have to worry either way. For the animators out there, it's getting so easy to make decent content, that market forces may solve this problem before the laywers have to. Witness the demise of BlockBuster Video in the era of digital music/video (Itunes/NetFlix, and even Netflix has re-birthed itself from its initial incarnation)...

cheers,

--ms

(edit = typos, and missing key word/idea)

fastbike1

@callmepaul

I suspect you won't like this answer, but you're probably (a guess since you didn't supply much info) having the same issue that many of the forum have . . .  complicate scenes and/or inadequate hardware.

Rendering isn't a game for the faint of hardware.

callmepaul

mindsong said:

Mechanically, using a rendering engine that's more open/portable/available like luxrender to actually execute the remote renders seems more plausible to me than trying to get 3DL or Iray renderers lined up on arbitrary remote hardware services like AWS (considering both setup compatibility and licensing issues). Taking the luxrender route implies using DS/Poser/Carrara to generate luxrender rendering data that can be 'sent' to the chosen luxrender-farm (via luxus/reality/luxus-core/etc.).

FWIW, @callmepaul, I wonder if your assertion that you 'own' the remote renderers for the duration of their use would hold up legally. I think it might, especially as this geometry duplication is merely a multiplier of the already sanctioned processing and still clearly consistent with the intended end-user/purchaser benefit. This, as opposed to the kind duplication/distribution involved with using these mesh products in legitmate game publishing or simply illegal pirating distribution - the main difference being in the second case, *other* users are in control and possessing the (rightfully) copyrighted mesh information, rather than a person running local and/or remote renders for his/her own use - generating a non-mesh related result (one or many images) - as was clearly intended in the sales model.

To wit, I think the argument that a render being done down-the-hall differs in any notable manner from those that would happen across the net, so long as the information involved is completely non-public and under the purchaser's control for the duration of the processing, would NOT hold up if challenged. In deference to the true copyright holders, I also don't think such a judgement would have a notable impact on either party's (vendor/user) financial situation either (damages). Who would be hurt if an animation using, say, 20 copyrighted digital meshes were completed on one machine in 30 days, vs 30 machines in 1 day? I don't see the damages that would validate a lawsuit, so long as the data were controlled, transient, and never publicly available. (But I'm no lawyer..., I periodically suffer that rare malady called common sense, which probably has no place in such conversations...). At best, I could see needing 30 legit copies of a product if the EULA differs from the 'like-a-book' usage model (many copies are OK, but only one active use at any moment), But what if the user has a single free version (e.g. G3F that comes with DS) of a mesh. Do they have to re-buy/register 30 free copies of DS to use it that way, etc.? And... does this licensing constraint hold up legally, even if most people naturally honor/respect/follow it? Interesting questions in interesting times.

To be absolutely clear, I completely understand, follow, support, and respect the copyright and terms of all content producers and distributors (my legit and way-too-large DAZ product library speaks to this).

As this particular form of rendering duplication is both controlled (kept completely within the user's operating domain) and is unambiguiously consistent with the original intent, sales pitch, and advertised use of the original products (products licensed for end-user rendering use), does the act of adding efficiency for the purchaser, with no measurable expense to the copyright holder (that I can see) change the nature of the relationship enough to nullify the original contract? I wonder how the legal system would (or will) rule on such use.

I'm certain many conference-room conversations have been had at DAZ/Rendo/RDNA/etc. about this very topic with lawyers and CFOs involved, It's clear that without good copyright protection this industry would not exist/survive. I'm not sure 'punishing' legitimate customers for using these various available efficiencies will work to the industry's benefit - witness the whole Connect fiasco. I would think that more cool animations coming out of the customer-base would mean more users getting drawn into the business and more of them buying content.

The good news is that this is all interesting stuff that fortunately doesn't impact most of the non-animator market-place, so most folks probably don't have to worry either way. For the animators out there, it's getting so easy to make decent content, that market forces may solve this problem before the laywers have to. Witness the demise of BlockBuster Video in the era of digital music/video (Itunes/NetFlix, and even Netflix has re-birthed itself from its initial incarnation)...

cheers,

--ms

(edit = typos, and missing key word/idea)

I am no lawyer either, and as yourself I try to live/work by common sense (manny times I realize that is not the best of ideas, but that might be for another forum). At this point to be honest I am more interested in the technicall stuff than the legalities. Not because I don't care about the legal stuff, but why bother if it is not techically possible in the first place.

I just found out about the NVIDIA release of iRay server (http://www.nvidia.com/object/iray-server.html) I have no idea how I can use it with DAZ, but the idea of an iRay Server looks promising at least.

Another point in my favour (not that I am counting) is that I see a DAZ Cloud in the advanced tab of the rendering section in DAZ. It says Beta. I am wondering what is that and how can I use it.

fastbike1 said:

@callmepaul

I suspect you won't like this answer, but you're probably (a guess since you didn't supply much info) having the same issue that many of the forum have . . .  complicate scenes and/or inadequate hardware.

Rendering isn't a game for the faint of hardware.

Thank you Fastbike. You are right, I don't like it hahaha, but that doesn't mean I won't accept it. But (there is always a but) I know there must be a solution out there for users like myself. Not big enough no justify spending a lot of money on a system designed to be able to render complicated scenes, but at the same time not small enough not to need to be able to render a complicated scene. 

I wonder if that DAZ Cloud (beta) tab is the solution...

mindsong

I like your pragmatism.

heh, The only real useful takeaway of my long-winded comment is that luxrender might be a portable/cloud-viable option and scene setup is available to us via both 'Reality' for poser/DS users, and 'Luxus' for DS/Carrara users. Not sure how easy it would be to export the kind of image series information that you could distribute across an AWS or RackSpace-like array of luxrender clients - can you export 'per-frame' directives, etc.

I wonder how many folks would buy an 'instant' render from a DS-based cloud service if the button were there to do so? And how much would they pay per frame?

cheers,

--ms

mjc1016

An easy way to find out, with Luxrender...install it on every machine in your home and start a network session.  Luxrender is the easiest renderer around to set up for network rendering.

callmepaul

mindsong said:

I like your pragmatism.

heh, The only real useful takeaway of my long-winded comment is that luxrender might be a portable/cloud-viable option and scene setup is available to us via both 'Reality' for poser/DS users, and 'Luxus' for DS/Carrara users. Not sure how easy it would be to export the kind of image series information that you could distribute across an AWS or RackSpace-like array of luxrender clients - can you export 'per-frame' directives, etc.

I wonder how many folks would buy an 'instant' render from a DS-based cloud service if the button were there to do so? And how much would they pay per frame?

cheers,

--ms

Definetely will text luxrender. Regarding paying for a render, I would definetely be a customer (at reasonable prices). I know how much I would have to pay for an EC2 instance to render my scene, so anything close to that would work for me. I wonder if that is what the 'beta' tab is there for...

mjc1016 said:

An easy way to find out, with Luxrender...install it on every machine in your home and start a network session.  Luxrender is the easiest renderer around to set up for network rendering.

Thanks a lot @mjc1016. I actually never made a render with Luxrender so I will give it a try and then test out the network session.

prixat

callmepaul said:

I wonder if that DAZ Cloud (beta) tab is the solution...

That's the (partially working) client for the Iray server.

The server uses web protocols for connection, which means it doesn't matter, to the client, if the Iray server is running on the same machine, on the same network or on DGX-1 boxes on the other side of the world.

With GPU memory sharing now possible, I'm pretty sure we will see Iray renderfarm services start appearing.

callmepaul

prixat said:

callmepaul said:

I wonder if that DAZ Cloud (beta) tab is the solution...

That's the (partially working) client for the Iray server.

The server uses web protocols for connection, which means it doesn't matter, to the client, if the Iray server is running on the same machine, on the same network or on DGX-1 boxes on the other side of the world.

With GPU memory sharing now possible, I'm pretty sure we will see Iray renderfarm services start appearing.

Thanks a lot, I just found out about the iRay server. Techically if I buy the license from NVIDIA and install the server on an EC2 instance then I should be able to connect from my local machine to my EC2 instance through the DAZ Cloud Tab?

That would be pretty interesting, do you have any experience with it? 

Thanks again

prixat

In theory that should work. I've only used the server on a local windows machine.

Calling on P2 instances (with 12GB K80 Teslas) sounds interesting.

(The 'partial working' side of things is not having the Queueing ability of the client enabled, only the interactive.)

Richard Haseltine

SimonJM said:

I suspect the answer would be no, as I thik you'd need to also upload the underlying geometry of the objects which is not permitted under the EULA.

We checked this with Daz - as long as the content is uploaded to the render service only for as long as it is needed to do the render, and as long as it is not during that period available to others (a private, temporary, copy) then the use of remote render services is permitted under the EULA.

mindsong

Richard Haseltine said:

SimonJM said:

I suspect the answer would be no, as I thik you'd need to also upload the underlying geometry of the objects which is not permitted under the EULA.

We checked this with Daz - as long as the content is uploaded to the render service only for as long as it is needed to do the render, and as long as it is not during that period available to others (a private, temporary, copy) then the use of remote render services is permitted under the EULA.

Now that is excellent news and the 'right' answer. I believe many folks don't know this to be the case, so I'll be sure to direct them to that as well. Is there a thread that can be used as reference (probably containing the original question, answer, and varations on the theme)?

excellent news, that.

--ms

Ati

With the legal question out of the way, does anyone have experience with rendering on AWS?

Kevin Sanderson

Thanks, Richard! That's great to know.

Ati, I have never used it, but I remember reading at Amazon that the GPUs they use are 4GB, so many would hit the memory wall.

Ati

Kevin Sanderson said:

Thanks, Richard! That's great to know.

Ati, I have never used it, but I remember reading at Amazon that the GPUs they use are 4GB, so many would hit the memory wall.

This page: https://aws.amazon.com/ec2/instance-types/p2/ says they provide NVIDIA K80 GPUs. Those have 12GB memory. Their biggest instance has 16 of these GPUs, although that costs $14.4 per hour. For 1 GPU, it's only $0.9 per hour, but I have no idea what that is capable of. And frankly, I'm not sure I even understand how the pricing of AWS works... :D I think you need to add storage to this and maybe even pay for network traffic, ... ?

Kevin Sanderson

Oh, good they upgraded! The pricing is a question to me, too. I figured I'd learn it doing some test renders eventually. Thanks for digging that up!

EDIT: the G2 instances for graphics applications are still 4GB GPUs. https://aws.amazon.com/ec2/instance-types/#g2

a_aa91787b

Old thread I know but if anyone comes by this post while searching for a solution, here's what I did:

I’ve setup an instance on GCP last month with Nvidia and Windows Server 2019. The cost was 0.9€/hour and render time from 2 to 5min.

https://www.nvidia.com/en-us/data-center/gpu-cloud-computing/google-cloud-platform/

In my opinion, its only worth the shot if you’re creating your scenes on your main device and exporting them to the cloud.
https://www.sharecg.com/v/91237/browse/8/Script/Daz3D-Scene-exporter-Standalone

On their page its says they’re available in Kubernetes, I don’t know that much about containers but if there’s a possibility to render via the CLI on windows then you’ll be able to gain a lot of time = money.

https://cloud.google.com/run
https://github.com/ephread/Autodazzler