Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.Daz 3D is part of
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Just an additional FYI about Windows memory management here. If you run a 32-bit process on 64-bit Windows, the virtual address space will increase to 4 GB -- double what you would have on 32-bit. However, you could experience additional crashes if Carrara was not compiled with the LARGEADDRESSAWARE flag -- because this extra virtual address space could be perceived as "illegal" at the >2 GB ranges.
Why go to 64-bit processes on 64-bit hardware? Because then your process space increases to 8 TB - and that is an artificial limit imposed by Microsoft...64-bit address space can be far bigger than that still. So memory allocation errors on 64-bit is almost guaranteed to be exhaustion of physical RAM rather than exhaustion of virtual address space.
Before C8, in the old forums, there was a stickied thread about the Large Address aware issue and a link to a LUAA (or something) script. Mac OS X was already large address aware, so we didn't have this issue.
Hello again.
Looks like this is going to be a longer thread. ;)
Well, we are living in 2013 not in 2003 anymore. V4 has around 70,000 vertices and around 60,000 polygons. Let's assume we are using 32-bit-floats for the vertices and for their vertice normals and 32-bit-integers for the polygon indexes which make up the polygons and let's assume further V4 is solely made of 4-sided polygons (just to push the memory boundaries).
vertices: 70,000 x 4 byte (32 bit floats) = 280,000 byte
vertice normals: 70,000 x 4 byte (32 bit floats) = 280,000 byte
polygons: 60,000 x 4 byte (32 bit integers as indexes for the vertices) x 4 (sided polygons) = 480,000 byte
That's ridiculous low 1,040,000 or roughly only 1 Mbyte to represent the figure in memory.
Now on with the shaders. Let's assume I have textures with the size of 3000x3000 pixel. Let's also assume I need roughly 6 textures for the whole figure (skin textures, eyes, eyelash, hair, some clothing). Some textures are usually smaller (eyes, lashes) so I think we are not bad with 6 textures in the size of 3000x3000 pixel all in all. Let's assume Carrara internally uses 4 byte for each pixel (RGBA) of the texture. We get:
6 textures x 3000 x 3000 x 4 (RGBA) = 216,000,000 byte
This is again far away form any 2 GByte boundaries.
BUT (!) when we take into account the number of »Undos« Carrara has to keep track off and the number of changes I made to shaders we can soon come to a limit. If there is a glitch in the internal memory management of Carrara this could be pretty much the reason for the Carrara behaviour.
Again: Good point again on the number of »Undos«.
Very nice image btw. :-)
Kind regards,
The Inflater
A few observations...
Your calculations, while reasonable for basic math, are far short of Carrara's internal usage. Carrara is a painfully academically correct C++ application built on top of a home-brewed COM-like reference counted system. Each vertex is a heavyweight object, implementing I3DExVertex (and a host of other interfaces). It is contained in a list in an IPolygon, in a list inside an IPolymesh, wrapped with an I3DExVertexPrimitive... and that's ignoring the rigging and morphing data.
And while your shader calculation is closer to accurate, the internal texture representation is 16-bits per channel floating-point RGBA (8 bytes per pixel) stored (again) in a heavyweight object called TMCColorRGBA.
I just tested, and on a fresh system, loading V4 took my Carrara usage from 128 MB to 274 MB: that's 146 MB just to load a bare-bones V4.
Now, to your other point that the shader room leaks.
Yes, I believe it does - getting a reference counted system perfectly clean is a hopeless task. You will get the best results if you disassemble complex shaders from the bottom up, leave "auto-refresh" off, and run "Remove Unused Shaders" frequently because it does a lot more than just drop things from the Master Shader list.
Carrara is not a garbage collected system, it is a reference counted system, and the more you work in a way that minimizes dangling references, the less trouble you will have.
I doff my cap to you sir! Excellent explanation as only a plug-in developer and/or DAZ code monkey could do it. Bravo!
Thanks a lot for some in depth look at the internal data structures of Carrara.
You are right, I just did the same with a bare V4 exported from Poser as an .OBJ so no textures. I have lower values but it's way higher than my expectation.
Yes, I believe it does - getting a reference counted system perfectly clean is a hopeless task. You will get the best results if you disassemble complex shaders from the bottom up, leave "auto-refresh" off, and run "Remove Unused Shaders" frequently because it does a lot more than just drop things from the Master Shader list.
Carrara is not a garbage collected system, it is a reference counted system, and the more you work in a way that minimizes dangling references, the less trouble you will have.
Thank you for the FIRST real hint in this direction. My point is not Carrara does leak. My point is it obviously does corrupt the saved data.
Which makes Carrara kind of unreliable. Especially for professional work (meaning: time constraints and earning money). And that's where we are now: I don't care for the internals of Carrara. I just know something's going damn wrong with it. And that's from someone who really likes Carrara. And I know from other render systems this is not the usual case: Use a program and pray everything goes well. And this was the basic reason for me to ask (in the other thread) about a list of correct or non-corrected bugs. If the programmers are not able to warn the user about a failure about to happen they should at least clearly point out it can happen.
Kind regards,
The Inflater
I will add a few more :)
The scene is not only figures and textures. It's also light and shadow. Reflection, refraction, transparency. Plus antialiasing. Plus filters. Plus Depth of field, Motion blur and so on.
When I set all this option to minimum, Carrara does the render very fast and without any errors even if I have high-poly models (3xV4 + complicated room interior). When I set all this to maximum, even the simple cube (SIX polygons) need some time and memory to render.
As about jargon. People have a right to use it. I become bored when somebody give me a long and detailed advice in reply to short question and this advice contains the explanation
1)What the RAM is and how does it works
2)Why the rendering requires a lot of memory
3)How to check available memory and my system type (32/64 bit)
and so on...
If there are short and well-known terms to call particular things, it's much more easy for user to just read enough about them and for programmer/pro to just describe the problem solving in short terms. If the user doesn't undertand, he can always ask. Or check FAQ. They're called "Frequently asked questions", but in fact I call them "RRA" - Rarely readed advices. If the user will understand all the terms once, many pro's will save their time on explaining the obvious things.
As about problem itself. I did a lot of investigation and found:
1)Now I have 4 GB RAM instead of 2. Freezing became SLIGHTLY less often, though the memory expanded more than twice. Looks like it's a virtual memory issue.
2)Freezing is not "slowing down". It's suspension. You can't move the render screen and can't switch back if you'll press Alt+TAB. You have to kill the Carrara process... and it consumes not so much memory at that moment!
3)It happens not only during renders. When I move a lot of stuff in a short amount of time, Carrara freezes. I have to save the scene and restart Carrara before every render.
4)Carrara has an internal memory management error. Because every next render has a less chance to success. Looks like it doesn't free all the memory used for previous render.
5)The more time passed from the render start, the more chance to fail the render. I saw how the scene freezed at 16%. When I changed the settings to improve performance, render failed at 26%, 63% and finally succeed. One render failed on 92%, when it was processing the simpliest part of the scene... It means Carrara most likely has a memory management error, not just the lack of RAM.
On the other hand, I found some ways to reduce the amount of problems:
1)Set the Maximum undo level as low as possible. I rarely use more than 1-2, while the default value is 16. Imagine how much memory you need to store 16 changes of complicated objects, especially adding/deleting...
2)System reboot HELPS more than Carrara reboot! I had a scene that failed 3 times though I restarted the Carrara, but succeed when I reboot the computer. Looks like Carrara produce a lot of garbage...
3)Watch the lights count. When I played with the "Sun light", it made even the simpliest render very slow and unreilable, while the single standard light doesn't slow the render down too much.
4)Disable Multi Threading! When I render with 2 threads, it often fails. When I use only one thread, it is two time slower but has much more time to succeed. I prefer a slow succes than a fast failure :D
5)The most helpful: delete all the invisible objects. Normally, Carrara should "forget" about them during render. In fact, deleting them before render makes the process much faster and reliable. I have two version for every scene: "Scene.car" for example (where all the objects are in their places) and "SceneRend.car" where all invisible objects (for THIS particular camera view) were deleted.
And today I've found a VERY funny thing...
When the render "freezes", you can't finish it even if you'll wait for a couple of hour. If you'll switch to another application, you'll be unable to switch back to Carrara. But when I press the power button, Carrara CONTINUES the work! :-)
If I work with the scene, it asks me if I want to save the render. If the render is already started (and freezed), it continues! Unfortunately, after 3-4 seconds the system turns off so Carrara doesn't have enough time to finish the render. I think, there should be a possibility to let Carrara finish the job, save the result and then let the system to shut down...