An idea that could make a real breakthrough in AI
eliezerbh_813cdbc78b
Posts: 71
If it were possible to create a character based on the facial features and body of a character with a certain name from your content, as well as give her a garment from your huge content, as well as hair and shoes and poses and facial expressions, everything comes from the huge content that daz offers to its customers, this would be a wonderful and unique thing that does not have the possibility This in no other AI.
Also, regarding the selection of the environment, the time and the weather and again from the content offered here on the site. This will enable some important and unique things that no other AI has.
Keeping a character exactly the same throughout several identical scenes which will open up the possibility for creators of comics and stories that require successive scenes of identical characters and environment.
And later perhaps develop a character with specific ethnic motifs that are not found on the site such as appearance, hairstyles and clothing that are used in all kinds of countries and peoples or periods and according to these guidelines create the plot that corresponds to the period, the country and the culture of the clothing and appearance of the people living in it.
No one has done this yet in AI and as we will see DAZ AI can do it more quickly because of the existing content.
think about it !
I would love to hear reactions to my idea if it is even a good and practical idea?
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
These things were already discussed when AI was just starting to become more mainstream in November 2022
But no one did anything with it!
Because of the way these systems work, what you're asking for isn't that easy to achieve. DAZ would need to build additional models called LoRAs ("low-rank adaptation") for each of the pieces of content that you might want to use, and use them to condition the output by combining them with the main model. In fact, that's probably exactly how their "Michael 9" and "Victoria 9" options work: they've built LoRAs trained on images of Michael and Victoria, and when you ask for Michael or Victoria, they add the appropriate LoRA into the generation process.
They'll probably add more over time, but creating LoRAs for every piece of DAZ content available would be very costly. Training AI models is expensive and requires non-trivial amounts of time and computer power.
Again, that probably involves building custom LoRAs. I would guess that if they do this then 'time of day' and 'weather' might be the first things they'd add, because those probably have the widest application. Developing LoRAs for specific environment content would have the same issue as developing one for a specific character: there's a lot of content, and you'd need to decide which pieces you want to build the LoRA model for.
There are other issues too. Suppose that DAZ decided to build a LoRA based on content from a specific PA, say Stonemason. You could build a training set from all Stonemason's sci-fi sets, for example. That would give you (if everything went well) a LoRA that could be used to create sci-fi images more or less "in the style of Stonemason". If I were Stonemason, I don't know how I'd feel about that.
So DAZ would have, at the very least, some tricky licensing issues to resolve.
This is something that the latest versions of the generators are starting to add: the ability to create character or setting consistent from image to image. I haven't played with any of the most recent iteration, so I don't know how well it works. My guess would be "sort of"; there's always going to be some drift from one image to the next, some of which may be almost unnoticeable, some of which may be glaring. This is actually an area where, again, a custom LoRA based on a specific character would help.
For these motifs, you need input images to use in your training set. Lots and lots of images. Where are those images going to come from? How are they made?
My best guess is that DAZ has basically taken the Stable Diffusion engine (and probably one of Stable Diffusion's bigger models, such as the one built from the LAION-2B data set) and is customizing the output by applying assorted LoRAs trained on DAZ content in order to give the results a specific "DAZ-like" look and to influence the human figures to resemble specific DAZ figures (Michael and Victoria presently). I would guess that they worked with Stability.AI's engineers to build training sets based on images made with DAZ content, including their flagship human figures.
They can probably automate the process of creating training images based on specific pieces of content to some degree. It's quite likely that they already did that for Michael and Victoria, so they may have a set of tools that will take any figure and render out large numbers of images of that figure from different angles, different lighting conditions etc. Those images then get fed into the training process to create the LoRA models.
If I'm right that they've automated that part, then they have an advantage. But it's a step up, not a magic bullet. They still have to pay the cost of (a) rendering all those input images, and (b) computing the LoRA from that. Both those parts are computationally expensive -- which means that they're costly in real world terms. The results are also not guaranteed; there's always the chance that they'll find that particular models don't tend to reproduce well in this way.
Unless Studio AI proves to be very profitable to DAZ, they'll probably only do it for a quite limited range of content. The economics are just against doing it on a very large scale.
Thank you very much for the detailed answer.
And what emerges from it is that AI is not what we thought, but rather what it thinks and we need to adapt ourselves to it. I don't see it as great wisdom.....
By the way, the midjourney AI command for maintaining a uniform character is cref, but again ineffective because it only knows how to maintain the uniformity of the character and not the uniformity of the environment in which the character is located, which again does not allow a sequence of consecutive identical scenes
they can do embeddings too
(Textual Inversions)
I made embeddings of my face and other things for Auto 1111 on a 2080Ti, it even had VRAM to spare and took about a day
(underclocked so not even much electricity)