Renders made real (sort of) with Deepfake Tech!

2

Comments

  • WonderlandWonderland Posts: 6,891

    Reality has become completely malleable. Here we are creating fake people as art, now people can create completely fake videos with celebrities and politicians. No one will know what’s real and what isn’t which just seems like a perfect extension of everything about 2020... The human species is doomed.

  • kyoto kidkyoto kid Posts: 41,062

    This reminds me of this, LOL

    ...+1  (BTW love Jeff Goldblum [and hate the site software as I always have to log back in after being in the store])

  • kyoto kidkyoto kid Posts: 41,062
    edited August 2020

    ...think I'll pass on this, I feel I've done well enough on my own 

    Leela shypw.jpg
    802 x 1322 - 459K
    Post edited by kyoto kid on
  • CypherFOXCypherFOX Posts: 3,401

    Greetings,

    It's not quite as fatalistic as that.  It's not terribly difficult to catch them right now; especially with certain motions.  And while I'm idly interested in audio GAN's, I'm not convinced anybody's got a good one yet.

    Eventually it will be possible to make ones that are very hard to detect, but even then, it will be possible to prove that a person wasn't there when the recording supposedly happened, for example.  It means that fact-checking is even more important.

    There is already a significant underground of people putting famous actors and actresses onto medium length porn videos, but willfully suspending disbelief is a key part of that medium.  Politics is much, much harder.  Because you have to make it plausible, and difficult to show the falsehood of.

    Nonetheless, we will survive when it's a little harder to tell if that outrageous thing is really something someone said, or if it's fake.

    I prefer to focus on the use that makes it possible to create really interesting art, that doesn't suffer from some of the uncomfortably 'not quite there' problems that too much 3D art does.  If you've seen a program named "GauGAN" (put it in quotes, or you'll get the artist, who's pretty awful as a person), it provides a really interesting direction for some of this.  You can sketch with 'concepts' and it draws them as things, based off of its database of images of that concept.  So drawing a tree in sketch form, fills in as a realistic looking tree in the output.  If, instead of raw sketches, you could provide a render, and the networks behind it used a large sampling of images to 'improve' the image...in the same way you can 'unblur' an out of focus image using a GAN.

    I wish I had the time and math knowledge to try and build it myself...it'd make a hell of a DAZ plugin. ;)

    --  Morgan

     

  • I wish I had the time and math knowledge to try and build it myself...it'd make a hell of a DAZ plugin. ;)

    It'd be the ultimate plug-in! "Real-ify."

    I don't have any real knowledge of GANs except a very basic idea of how they work. But it would seem to me this would be totally within their wheel house.

  • Rashad CarterRashad Carter Posts: 1,803

    Okay, so this is kind of weird but also kind of cool.

    I’ve been messing around with “deepfake” tech and I decided to see what would happen if I tried to create a deepfake with one of my rendered characters. The results? The AI sort of made her into a real person! It plugged her into TV shows and movies.

    I really don’t know what (if anything) I can do with this, but I thought it was amazing how the AI could take her face and make it completely photorealistic. I wonder if there is some kind of application for that in rendering? Like, could an AI algorithm "real-ify" our images if it was trained?

    Here's the original image I uploaded:

    And here is the output:

    Mixed feelings like most everyone at the moment. Must note how darned beautiful she is! Yes, getting all that realism from direct renders would be nice. Fascinating for sure! This AI is quite intelligent and skilled.

  • nonesuch00nonesuch00 Posts: 18,142
    Nath said:

    Does anyone know of something similar to reface that runs on a computer rather than a phone?

    https://github.com/iperov/DeepFaceLab/

  • kenshaw011267kenshaw011267 Posts: 3,805
    CypherFOX said:

    Greetings,

    It's not quite as fatalistic as that.  It's not terribly difficult to catch them right now; especially with certain motions.  And while I'm idly interested in audio GAN's, I'm not convinced anybody's got a good one yet.

    Eventually it will be possible to make ones that are very hard to detect, but even then, it will be possible to prove that a person wasn't there when the recording supposedly happened, for example.  It means that fact-checking is even more important.

    There is already a significant underground of people putting famous actors and actresses onto medium length porn videos, but willfully suspending disbelief is a key part of that medium.  Politics is much, much harder.  Because you have to make it plausible, and difficult to show the falsehood of.

    Nonetheless, we will survive when it's a little harder to tell if that outrageous thing is really something someone said, or if it's fake.

    I prefer to focus on the use that makes it possible to create really interesting art, that doesn't suffer from some of the uncomfortably 'not quite there' problems that too much 3D art does.  If you've seen a program named "GauGAN" (put it in quotes, or you'll get the artist, who's pretty awful as a person), it provides a really interesting direction for some of this.  You can sketch with 'concepts' and it draws them as things, based off of its database of images of that concept.  So drawing a tree in sketch form, fills in as a realistic looking tree in the output.  If, instead of raw sketches, you could provide a render, and the networks behind it used a large sampling of images to 'improve' the image...in the same way you can 'unblur' an out of focus image using a GAN.

    I wish I had the time and math knowledge to try and build it myself...it'd make a hell of a DAZ plugin. ;)

    --  Morgan

    That's the kind of scary part. You don't need that much math. The code libraries already exist. You could put together a GAN with a fairly basic level of coding skill (it might not be a great GAN but as long as it worked at all it would still start training and that's the whole point.

    3 years ago my datacenter was mostly databases and websites. We had, IIRC, one cabinet doing AI training. Now it's more like half the center and its only half because we can't get the Quadros for more than that. GAN's don't yet, to the best of my knowledge, have commercial applications but other forms do (although I personally think the ones being used to "predict" the stock market are bogus).

     

  • maikdeckermaikdecker Posts: 2,752

    While cool tech, the legal ramifications with real people is scary to me. We are a society were everyone has a camera on them 24/7 and as such we rely on videos to fact check more than ever. Social media already guns the gun and makes everything, whether true or not viral. Can't see this helping.

    Relying on videos to "fact check" was no option since about 20 years ago already. And relying on social media to "fact check" never was an option.

    Too bad that too many people still think it would be... devil

  • Okay, okay, last one I promise.

    This time she's Margot Robbie. Personally, I think she's prettier. wink

     

  • Fascinating work Emoryahlberg, thanks for sharing.

    It's kind of like the opposite of Hollywood stunt stand-ins. In that case, they film the stunt double and then map the star's face onto them. But you've a got a clip of a famous person doing something and are mapping a fictitious face onto the star. Interesting.

  • TaozTaoz Posts: 9,943

    This reminds me of this, LOL

    +1

  • duckbombduckbomb Posts: 585

    While cool tech, the legal ramifications with real people is scary to me. We are a society were everyone has a camera on them 24/7 and as such we rely on videos to fact check more than ever. Social media already guns the gun and makes everything, whether true or not viral. Can't see this helping.

    Relying on videos to "fact check" was no option since about 20 years ago already. And relying on social media to "fact check" never was an option.

    Too bad that too many people still think it would be... devil

    This.

    Its my opinion that we need to stop trying to slow the inevitable spread of tools that spread the misinformation and start addressing the "culture" of misinformation spurred on by social media and fake news.

    This is cool.  The tech isn't what scares us, but what people will do with it and (even more so) what they'll believe.

  • TaozTaoz Posts: 9,943
    duckbomb said:

    While cool tech, the legal ramifications with real people is scary to me. We are a society were everyone has a camera on them 24/7 and as such we rely on videos to fact check more than ever. Social media already guns the gun and makes everything, whether true or not viral. Can't see this helping.

    Relying on videos to "fact check" was no option since about 20 years ago already. And relying on social media to "fact check" never was an option.

    Too bad that too many people still think it would be... devil

    This.

    Its my opinion that we need to stop trying to slow the inevitable spread of tools that spread the misinformation and start addressing the "culture" of misinformation spurred on by social media and fake news.

    This is cool.  The tech isn't what scares us, but what people will do with it and (even more so) what they'll believe.

     - only two things are infinite, the universe and human stupidity, and I'm not sure about the first.
       -  Albert Einstein

  • nonesuch00nonesuch00 Posts: 18,142
    edited August 2020

    Faceswap is the one I messed with back when I did:

    https://faceswap.dev/

    I'm sure there are others.

    Warning. When I tried to download that product Windows 10 Windows Defender blocked it saying it could harm my computer.  The massive 14.5GB Deep Face Labs zip file from Mega you can find on this page however has no such warning:

    https://github.com/iperov/DeepFaceLab/

    As I only have a nVidia GeForce GTX 1650 Super 4GB & 32GB system RAM I don't know if I'll mess with it anytime before next spring though. 

    Post edited by nonesuch00 on
  • OOOOH for DAZ or better yet for Blender with EVEE!!!!!

  • droidy001droidy001 Posts: 282
    edited August 2020
    Probably about 5 or 6 years ago I was having a conversation with mate about CGI movies. How very soon it would be indistinguishable from the real thing. We are getting close. My thought was that the actors would play the roles, this would then be converted to cgi. You would could watch the movie from anywhere on the set. With the advance of real-time rendering, Ray tracing and VR the possibilities are exciting. I know they do real life VR movies now but this is restricted to 2 or 3 camera angles so that the lighting and cameras etc are not in view. With lifelike cgi that is not a worry.
    Post edited by droidy001 on
  • nicsttnicstt Posts: 11,715

    This reminds me of this, LOL

    + 0.5

    Why?

    Technological changes and scientific advances always come with both a cost and a benefit.

    It's up to us to mitigate the cost, and amphasise the benefit.

     

  • nonesuch00nonesuch00 Posts: 18,142

    For those that think this is a paid mobile phone app or nVidia GPU only option at the link of the Deep Face Lab link I posted above: there is an openCL EXE self-zip file named DeepFaceLab_1.0_OpenCL_build_01_11_2020.exe in the "DeepFaceLab 1.0" folder that runs on AMD GPUs and yes, slow, but it would work on intel GPUs as well. You want to use the nVidia version for nVidia GPUs although technically openCL works there as well.

    Also to cut down on the 14.5GB if you wanted to:

    You need to download only the zips inside the Facesets folder, the zip in the PretrainedModels folder, there is only one AMD/openCL version DeepFaceLab_1.0_OpenCL_build_01_11_2020.exe or the nVidia version you want to use DeepFaceLab_NVIDIA_build_08_02_2020.exe being the latest available compiled nVidia version (Aug 02 2020).

    If you compile it yourself you have many auxilary libraries and tools which you have to compile and install. They are all free but you will get an education in build chains.

  • SevrinSevrin Posts: 6,308
    nicstt said:

    This reminds me of this, LOL

    + 0.5

    Why?

    Technological changes and scientific advances always come with both a cost and a benefit.

    It's up to us to mitigate the cost, and amphasise the benefit.

     

    Yeah, that doesn't always work out.  In this case, there's not a lot of societal benefit compared to the potential societal cost.

  • PerttiAPerttiA Posts: 10,024

    One thing came to mind as I was browsing the forum... Apparently the technology uses thousand(s) of photos of the "victim" to make the illusion as good as possible in any angle and/or emotion - Couldn't the same kind of method be used in creating better likeness when creating a 3D model of that "victim" as well?

  • PerttiA said:

    One thing came to mind as I was browsing the forum... Apparently the technology uses thousand(s) of photos of the "victim" to make the illusion as good as possible in any angle and/or emotion - Couldn't the same kind of method be used in creating better likeness when creating a 3D model of that "victim" as well?

    I think that is typical, although with this software (which only creates GIFs), you only need one image.

  • mmkdazmmkdaz Posts: 335

    Really cool. Surpases all my expectations

  • hjakehjake Posts: 896

    Reality has become completely malleable. Here we are creating fake people as art, now people can create completely fake videos with celebrities and politicians. No one will know what’s real and what isn’t which just seems like a perfect extension of everything about 2020... The human species is doomed.

    Sorry but fake people have been around for a long time. Although they weren't very deep. This is just going the last mile and bringing them into the digital world as the vanguard before we all move there.  cheeky

  • maikdeckermaikdecker Posts: 2,752
    hjake said:

    Reality has become completely malleable. Here we are creating fake people as art, now people can create completely fake videos with celebrities and politicians. No one will know what’s real and what isn’t which just seems like a perfect extension of everything about 2020... The human species is doomed.

    Sorry but fake people have been around for a long time. Although they weren't very deep. This is just going the last mile and bringing them into the digital world as the vanguard before we all move there.  cheeky

    Expect me to sit in my comfy chair and wave you all - those who go digital - a friendly bye bye...

    More bodyless digital humans in the digital world means more steak for me devil

  • SevrinSevrin Posts: 6,308
    hjake said:

    Reality has become completely malleable. Here we are creating fake people as art, now people can create completely fake videos with celebrities and politicians. No one will know what’s real and what isn’t which just seems like a perfect extension of everything about 2020... The human species is doomed.

    Sorry but fake people have been around for a long time. Although they weren't very deep. This is just going the last mile and bringing them into the digital world as the vanguard before we all move there.  cheeky

    I thought we were going to Mars.

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,230

    there is no spoon

  • Space may be the final frontier, but it was made in a Hollywood basement.

  • PerttiAPerttiA Posts: 10,024
    hjake said:

    Reality has become completely malleable. Here we are creating fake people as art, now people can create completely fake videos with celebrities and politicians. No one will know what’s real and what isn’t which just seems like a perfect extension of everything about 2020... The human species is doomed.

    Sorry but fake people have been around for a long time. Although they weren't very deep. This is just going the last mile and bringing them into the digital world as the vanguard before we all move there.  cheeky

    Expect me to sit in my comfy chair and wave you all - those who go digital - a friendly bye bye...

    More bodyless digital humans in the digital world means more steak for me devil

    But then again... Since everything you percieve through your sensors is transferred to your CPU as electrical signals. How do you know that those signals haven't been tampered with? Maybe they are completely artificial, generated by some pimple faced alien playing with his imaginary friends.

    Going down the rabbithole, only one thing remains... "I think, therefore I am"... Somewhere in the universe exists an entity in some shape or form that can call him/herself "Me" due to the fact that it is capable of thinking and everything else is just figments of this beeing's imagination.

    The steak you think you see, is nothing but a bate, take a nibble and see what the beeing has conjured up for you cheeky

  • CybersoxCybersox Posts: 9,059

    Okay, so this is kind of weird but also kind of cool.

    I’ve been messing around with “deepfake” tech and I decided to see what would happen if I tried to create a deepfake with one of my rendered characters. The results? The AI sort of made her into a real person! It plugged her into TV shows and movies.

    I really don’t know what (if anything) I can do with this, but I thought it was amazing how the AI could take her face and make it completely photorealistic. I wonder if there is some kind of application for that in rendering? Like, could an AI algorithm "real-ify" our images if it was trained?

    Here's the original image I uploaded:

    And here is the output:

    The thing is, the reason that they look so much better is because the Deep fake tech completely scraps the way that DAZ figures make facial expressions, with the DF techniques mapping a shape pattern over a face that is actually bending and posing in a completely natural manner.  Unfortunately, that can't be piped back into studio to the figures because DAZ figures simply don't have the kind of facial flexibility to directly import those expressions, especially the Genesis 8 lineup with it's over reliance on facial bones, because most real facial muscles create expressions via overlapping layers of influence rather than revolving around a fixed set of pivot points.  Now, if you're willing to completely scrap all of the bones and instead import expressions that are primarily created by whole-face morphs, you might be able get something similar.    

Sign In or Register to comment.