Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.Daz 3D is part of
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
There is no fitting happening when loading the Dev Load version of G8 into an empty scene, which is the only reliable way to compare the loading times of the figure.
Answer for Q1: It depends on the content of the morph file. What really causes the slowdown is the number of formulas (dials in posing and shaping pane) Daz has to load. Morph files can contain both formula info and geometry info. Geometry info is loaded on demand, so it has a minimal impact on load time. At character loading time, Daz parses all formulas from text and convert to Daz internal representation, this process seems to be what is causing the slowdown. Daz caches all the formulas for a figure but still store it in text format hence why SSD doesn't help much. But Daz still check the metadata for each morph file so disk access time still have some impact on loading time.
Also, some morph files are compressed, so they appear to be smaller than they are.
Q2. Some characters are morphs that include tons of formulas, so they are a major culprit for the slow loading time. Other characters are just preset or geometry data that shouldn't impact loading time. Other stuff that impact loading data include poses (if they got sliders), expression, morphs set, clothing that have corrective morphs (like shoes).
Can you explain a bit more? If what you say is true, Genesis 8 Dev Load file should load instantly because it only contains the base figure with no eyelash/clothing/hair. I do acknowledge that it might be possible that both loading morph and fitting might be slow because I have not measured fitting speed.
Long load times are down to a few things happening at the same time, and giving DS fits as it tries to work through them.
First up is the amount of DSF files in the figures "Morphs" folder, DS has to load and process every single file in there before the figure will appear in your scene.
Second is the number of formula (ERC in DS) in each file, DS has to create each and every one of those ERC links before the figure will load, and depending on what the DSF file is for, you can have anywhere from one formula per file to over a thousand formula a file.
Third is poorly made content.
Fourth is the fact that all of the above is being processed on a single core of your CPU, any issues with loading any of the above and you get a "bottleneck" as the data is struggling to be processed, which gives you long load times.
This should do what you are proposing and I think it still works (Vaguely remembering it kinda works despite it being an old tool when I tried it). The tricky part is knowing what you want to use in the future and make it easier to add more content to the scene. Which is why I made this.
On the technical side, the tricky thing is to find dependencies recursively. Otherwise, you just need a programming language that let you open gzip file, query JSON file, and maybe handle xml as well.
Coincidentally, I have implemented enough features in my program in the last few days that it almost does what you want. It can analyse files in a single folder and spit out the required Daz product and other files. The only things missing are recursively processed a folder and find transitive dependencies (like if product A requires product B).
C# definitely make implementing the program easy. Just have to poke at DIM install manifest folder and Daz metadata.
Not necessarily poor coding, but coding without foresight. It was probably acceptable at some point. OK, I suppose that's a specific case of a type of poor coding.
DS is not Open Source, so none of us know what is happening, and the current behavior could actually be not the worst case scenario; we shouldn't second-guess the devs as we don't know what kind of architecture they have been saddled with.
But as users, we are certainly entitled to say that we are having a poor user experience, regardless of the technical reasons for it, without anyone trying to minimize or justify. There is no excuse for an architecture that somehow allows other unrelated morphs to erroneously destroy your character, or to require you to install morphs that your character doesn't even use and that didn't even exist when you made the character.
Daz needs to fix this. Period. But I do not have super high hopes, because Daz also made an "architecture" that allows PAs to mess up metadata and poorly name things...
PA's do not make metadata, DAZ does.
I'm not sure what you mean about morphs destroying characters, unless it's the cases where a morph loads with a bad default value or the ERC is not correctly set up to limit the scope of a correction (both of which are good functions not correctly applied). I've no idea what you mean by needing to install unrelated morphs.
Aside from speed, which as you say may involve tradeoffs, I'm not sure what core-feature issues there are (as opposed to errors in specific products).
It really doesn't matter who. A framework that is doing its job would not allow this to happen.
I mean exactly what I wrote. As for destroying characters, a morph that I did not apply, that I hadn't even known existed, suddenly and silently gets applied to my character. I didn't discover it until after an overnight render that I had to do again. As for installing unrelated morphs, I recently had to re-install everything for an unrelated reason. While loading up my main protagonist, it prompted me to install several assets that the character does not use, and did not even exist when I created the character. Things that I bought, installed, decided that I didn't really like, and that are not displayed on the "Currently Used" tab.
I would certainly consider the issues I just described as core features. I repeat that an architecture that requires DS to evaluate all the morphs that are installed globally and not just the number of morphs actually deforming a mesh is just a bad architecture. Full stop. There is no way to explain this one away with technical sounding terms that don't help scenes load any faster by writing them in a forum.
Like @charles said, I don't see how this can not be affecting Daz's bottom line, what with the number of people complaining about the user experience. I am just hoping that v5 rectifies as much technical debt as possible... if they're going to break binary compatibility, then let them fricking break binary compatibility and implement as many long-standing and invasive fixes as possible in one shot, as opposed to subjecting the userbase to these bugs for even longer, just to have to break it again later anyway.
I'll be the first one to profess my affection for Genesis as a framework and the incredible PAs that make it shine. I really, really want Daz to continue to succeed. I am actually as content giving Daz and the PAs my money as I am the Blender Foundation. But things like this really make me worry about its viability in the future.
That is more of a problem with Daz QA. Daz definitely should do a better job at QA stuff they sell to make sure broken stuff doesn't get released. But the program's perspective, it is really hard to figure out what is broken. It can't read your mind and figure out what you think is broken. Although what would be useful is some tools for power users to make fixing stuff easier.
This goes to the above point. Daz loads every morph related to the figure in the base directories you have added. Even if it might not do something now, the user might want to use the morph later. For example, I have asked people if they prefer no expressions in the posing panel and have the figure load faster or the current situation and many people I have talked to said they preferred having expression sliders. To make it beginner-friendly, Daz loads all expressions when you load the figure so that the user can just go to an expression slider and use it to apply the expression.
Definitely agree that Daz should fix this. But disagree that they should fix this before releasing DS5. It is better to release a few features early than waiting forever for all features to finish. The fact is currently DS4 is broken for many users and DS5 is being released to fix that. Unless they already implemented a fix for character loading and are just waiting for a binary break, they should release DS5 and then work on a fix and release it later.
If you have files installed via DIM/Central, just point my program to the install manifest folder and get it to load all the products you have installed. Alternatively, you can add any folder as 3rd party folder. Then just get it to scan files and it will autoselect all products referenced in the scene and any 3rd party files it matches. You can then save it as a selection and it is just a JSON file with product names and file locations. You can edit the result into a format that DIM accepts and that should let you select all products used. Run this process on as many folders as you want before saving the list.
Edit: It appears DIM need "Product Name", "Product ID", and "Package ID" to make selection work. I can probably make it work for stuff sold through Daz store.
V 1.9 should work, but I have implemented a few more error handling and recursively scan a scene folder in V1.10 that is currently in the dev branch.
The program works for me, but it might still have many corner cases it is not handling. No one has told me if the current version works or not for them.
I disagree. It is not a job for a human being.
What you are saying is hard to do is only hard to do because there is no framework in place that could detect and prevent a large number of these mistakes. Consider this: if the product pages were automatically generated from some template, and everything else is created from that master template, the framework could automatically check to see that all the morphs included on the product page were actually in the asset pack. It could force them to be placed in the correct and consistent file locations. If it were an HD addon, it could enforce the name to be the original name with "HD" appended to the end, and make the product page always say so.
There is even a markup language called SGML that was designed to automatically create documents in a structured manner so that references, figures, footnotes, technical drawings, etc... always match up correctly no matter what is changed.
Even if I accepted that as a good idea, there should be a much lighter-weight method for doing that, that does not cause such strident complaints about the terrible user experience because of it. These are software engineers. They should engineer it. An explanation/justification of why my user experience is so bad is not what I want. I want a better user experience.
Fair enough, I see your point. Here is mine: as a plugin writer, I would prefer to be adding the new features that the people who use the plugin have asked for instead of re-reverse engineering the changes that Daz made and are no doubt not going to document. That sucks more than having a tooth pulled because the pain lasts longer. I'd like to do it as few times as possible.
Speaking of engineers, you cannae change the laws of physics (or coding). Not evey problem has an ideal solution, this may well be one where there are trade-offs and it isn't clear that the needs of those of us with large libraries should trump the needs of those who want convenient access.to their morphs (to the extent that the groups don't overlap anyway).
I'm not sure what you mean by "The Laws of Coding", but I really hope you meant Complexity and/or Algorithmic Analysis. I hope so, because then we can have an objective discussion. But then, we can only come to the conclusion that it's a bad architecture. Any rational discussion is going to somehow come back to that.
I'll go one step further and say that no non-trivial problem has an ideal solution, and there are always tradeoffs to be made, the most obvious being between time and space. And generally, the more complex the system, the more opportunites there are to work tradeoffs. It should never be the case where a system performs well under a certain set of conditions, and dismally in the others. If it does, bad architectural design choices have been made.
But I'm willing to suppose that the Daz devs are good at what they do. This is not an ad hominem attack. That's why I distinguish between a bad architecture and an architecture that is simply encountering unforeseen conditions that were not part of the initial system level requirements, and under which it doesn't perform well. You see this sort of thing all the time, and its a much more viable explanation than that Daz meant it to take 5-30 minutes to load a scene and thought that people would be happy with that.
A perfect example: years ago, around the 2.2 series, the Linux had always had O(log n) complexity scheduler. It worked like a champ when Linux ran on desktops with a relatively small number of runnable processes at any given time. But when Linux started to move into the enterprise space, it could spend more time trying to choose the next process to run than it did... running the next process. So they developed an O(1) scheduler because the O(log n) scheduler was written by Linus himself when Linux was his small, pet project and he had never dreamed of Linux even having a GUI one day, much less run mainframes with thousands of processes. Note that the Linux developers did not keep trying to explain to their users why Linux scaled so poorly, they ripped out the scheduler and wrote a new one more appropriate to the space in which Linux was now operating.
Would you agree that all the above is reasonably objective?
Yes, that is what I meant. Before calling ti bad architecture you need to offer, by your own design or by pointing to another application, a system that would satisfy what we know of the design specs in a "better" way (and you need to explain what better is, and why any downside is the lesser evil). You keep asserting that it's bad design and that the developers should fix the problem, but that is begging the question of whether the basic logic (ability to add new morphs, ability to access morphs in a conveinent manner) can be implemented in a way that is better; proof of possibility comes before finding fault.
This is all assertion, though.
Yes, it is indeed possible that there is more genesis 8 content than the original designers were expecting due to its unusually long lifetime, and i'm sure they are looking at future development in light of this. But we do need to remember that this is not Daz' first morph management system, as I'm sure I have pointed out before, there was morph injection in the third generation figures and ExP in the fourth - the daz team has much experience in thif field, and much feedback from users at all levels (not just those of us with large content sets).
I have to point out that the daz developers do not keep trying to explain anything - I keep trying to explain why it is not clear that their implementation is significantly sub-optimal given its objectives, and that there is no guarantee that it can be greatly improved, and that asserting that it just needs fixing is making unwarranted assumptions. I certainly hope that they are working on the issue, and that they can come up with a system that will make life easier for those of us with large cointent libraries; I am not making assumptions about their abilities, or about how any shortfall in the current or future systems may relfect on their skills or judgement.
"Hey, this problem has linear complexity against something that's unbounded. That is unacceptable." That's all it would have taken. As it stands, this is the worst type of conflict: The better Daz does at its main goal, selling assets, the worse the app will perform at its primary function. There is simply no way to present that as a good architecture, and also no excuse that can be made.
If you do not accept at least that much, then there's no point in debating further.
save your breath
--ms
I know, I know :) I know that the devs are aware of the problem, and surely want to fix it. I just can't stand the practice of invalidating, minimizing or dismissing the legitimate concerns of users as if after 33 years I didn't know a bad architecture when I see one.
I am not in anyway dismissing the problem - it affects me too. I am saying that you are not enttiled to call it bad architecture unless you can point to or provide a better. The slow loading can be an issue, and can be discussed, without the need to apportion blame.
That is a dismissal. Something need not be the worst of all possible manifestations before it can be said that it is not meeting expectations. Think. Literally close your eyes and imagine how the world would be if what you wrote were a universal rule.
One need not apportion blame when there is only one entity whose fault it could possibly be. Daz Studio does not grow like mold on bread nor is it delivered from the heavens on stone tablets; Everyone knows from where it comess.
No, it is not a dismissal. I am not disputing the impact it can have.
No. If you want the feature 9and it certainly seemed, over the eyars, that many users do in fact want the features that this system enables) then it is bad architecture only if there was a better way to implement those features. Even then, implementing it as it was might have turned out badly for those of us with large content libraries but I don't know that Genesis 8's four-year run as the flagship figues was foreseeable so I'm not prepared to say that going with the system they did was a poor choice (especially as it had served well throughout the previous Genesis figures' run, with the issue perhaps starting to crop up towards the end for Genesis 3 Female.
As I said, there is nothing wronf with pointing out the issue or with lobbying for a revision of the system or , if oen can be devised, a better system - but it should be donen in a way that does not make assumptions about what is possible given the design goals or what was foreseeable in regards to how far it would need to scale.
The load time for a G8F on my system increased from 30 seconds to meanwhile 8 minutes. I think DAZ has some very good software engineers. My suggestion is a tool that enables or disables the morphs/poses/etc. that are not needed.
We see some freeware here in the forum that shows that this is possible. I think it is no great a great expense for DAZ to create such a tool. And I think like me, many people are also ready to pay for that.
(Sorry for my english, that is not my native language).
Since DS does not know your "intent" when adding a G8F character, it would have to be some sort of "save file" I would think. Otherwise, it would be limiting all of the morphs and options you bought, that you may get the urge to dial-in. Once I create a character, if I like it...then make a saved preset that only loads that character's morphs. (of course, this then prevents you from tweaking it during that session...and do you realy want that?)
Later I decide I need a little Fitness morph in there...now I can't dial it in, cause it was not loaded. I have to find a version that I can open with morphs to load it, or go back to the old injection thing. This turns into a big can of worms where everyone has different needs and wants.
This is why I am happy with a method to just turn off things I rarely use in general. I still have the flexibility to tweak characters, and can always re-enable those impulse-buys, that are slowing me down in most cases.
...but thats just me