Loading...
> >

Renders made real (sort of) with Deepfake Tech!

emoryahlbergemoryahlberg Posts: 133
edited August 2020 in The Commons

Okay, so this is kind of weird but also kind of cool.

I’ve been messing around with “deepfake” tech and I decided to see what would happen if I tried to create a deepfake with one of my rendered characters. The results? The AI sort of made her into a real person! It plugged her into TV shows and movies.

I really don’t know what (if anything) I can do with this, but I thought it was amazing how the AI could take her face and make it completely photorealistic. I wonder if there is some kind of application for that in rendering? Like, could an AI algorithm "real-ify" our images if it was trained?

Here's the original image I uploaded:

And here is the output:

Post edited by emoryahlberg on
«13

Comments

  • NylonGirlNylonGirl Posts: 1,825

    I definitely want to do this.

  • NylonGirlNylonGirl Posts: 1,825

    Woud it be possible to use a video of myself as a base for the DAZ character?

  • emoryahlbergemoryahlberg Posts: 133

    Sure, if you used face transfer!

  • jesusaramenesjesusaramenes Posts: 177

    Good job, impressive !

  • ed3Ded3D Posts: 2,198

    _ interesting and Fascinating _

  • FSMCDesignsFSMCDesigns Posts: 12,755

    While cool tech, the legal ramifications with real people is scary to me. We are a society were everyone has a camera on them 24/7 and as such we rely on videos to fact check more than ever. Social media already guns the gun and makes everything, whether true or not viral. Can't see this helping.

  • NylonGirlNylonGirl Posts: 1,825
    emoryahlberg said:

    Sure, if you used face transfer!

    I asked that question the wrong way. I mean is it possible to use a personal video as a substitute for motion capture, or does it have some premade set of scenes that the animation uses to create these effects?

  • CypherFOXCypherFOX Posts: 3,401
    edited August 2020

    Greetings,

    My understanding is that those are just gifs that they pulled from the internet.  You could definitely use your own video/gif of yourself, if you're running the deepfake software yourself.  So you could, in theory, record yourself doing a Tik Tok dance, then render an image to use for your face, and then use the deepfake to replace your face with the rendered face.

    And the ramifications are...complicated.  But such has been the problem of technology in photography ever since National Geographic moved two pyramids closer to each other to make a better cover picture.

    --  Morgan

    Edit: On the flip side, I would _LOVE_ a technology that took a render of mine, ran a GAN against it, until the adversarial network couldn't tell that it was a render anymore...  Just 'real-ify' an image...

    Post edited by CypherFOX on
  • emoryahlbergemoryahlberg Posts: 133
    NylonGirl said:
    emoryahlberg said:

    Sure, if you used face transfer!

    I asked that question the wrong way. I mean is it possible to use a personal video as a substitute for motion capture, or does it have some premade set of scenes that the animation uses to create these effects?

    You absolutely can. I used a software available on phones called "Reface."

  • kenshaw011267kenshaw011267 Posts: 3,805

    This is a significant issue. While a single home computer would struggle to deepfake more than a gif or a tiktok video in a reasonable time I run a datacenter. I could, assuming I didn't mind getting fired when the auditors caught me, use a bunch of servers to deepfake a politcian saying pretty much anything. Someone with $100k could just rent a rack and do it legit which means someone is doing it right now. Also its only a matter of time before someone in some country where US ip laws aren't really a thing starts doing deepfake porn, actually that is probably already a thing but I'm not poking around on the really sketchy parts of the web to verify.

    Also look at the OP's images. With no idea how long it took to train the AI or how long it took to render the gif's there must be some producer in Hollywood starting to wonder why they are paying actors. 

  • FSMCDesignsFSMCDesigns Posts: 12,755
    kenshaw011267 said:

    This is a significant issue. While a single home computer would struggle to deepfake more than a gif or a tiktok video in a reasonable time I run a datacenter. I could, assuming I didn't mind getting fired when the auditors caught me, use a bunch of servers to deepfake a politcian saying pretty much anything. Someone with $100k could just rent a rack and do it legit which means someone is doing it right now. Also its only a matter of time before someone in some country where US ip laws aren't really a thing starts doing deepfake porn, actually that is probably already a thing but I'm not poking around on the really sketchy parts of the web to verify.

    Also look at the OP's images. With no idea how long it took to train the AI or how long it took to render the gif's there must be some producer in Hollywood starting to wonder why they are paying actors. 

    Agreed.

  • DaremoK3DaremoK3 Posts: 798

    That is both awesome and scary (in line with what FSMCDesigns said) at the same time.

    I just want to know if you hand picked the GIF's for the AI to work with, or did the AI pick what it thought best would fit the 3D model?

    Regardless...  Impressive  --  Especially with matching the mouth opening.

  • emoryahlbergemoryahlberg Posts: 133
    kenshaw011267 said:

    Also look at the OP's images. With no idea how long it took to train the AI or how long it took to render the gif's there must be some producer in Hollywood starting to wonder why they are paying actors. 

    Seconds. It was all done through an iPhone app!

    Here's my character as Black Widow:

    Again, I'm mostly impressed with how the AI made her photoreal, something we've been chasing for years! 

    I just wonder if there's some kind of AI trainer that could make our renders and make them photoreal.

  • emoryahlbergemoryahlberg Posts: 133
    DaremoK3 said:

     

    I just want to know if you hand picked the GIF's for the AI to work with, or did the AI pick what it thought best would fit the 3D model?

    I chose all ofthe GIFs based on what looked fun, and also because my character is blonde I mostly stuck with that... smiley

  • emoryahlbergemoryahlberg Posts: 133
    edited August 2020
    CypherFOX said:

     

    Edit: On the flip side, I would _LOVE_ a technology that took a render of mine, ran a GAN against it, until the adversarial network couldn't tell that it was a render anymore...  Just 'real-ify' an image...

    YES! I mean, doesn't my examples prove that must be possible? That's essentially what is happening. 

    Post edited by emoryahlberg on
  • WonderlandWonderland Posts: 6,891

    Wow! What app is that?

  • The Blurst of TimesThe Blurst of Times Posts: 2,410

    I think the pussycat is already out of the bag with deepfake porn.

    People have been photoshopping for a while, but I mean... if it's easy to marry an image to a gif? I bet it's being done, and has been done for a while.

    (Porn tech will always exceed our tech. poser/daz pr0n has been there for a while. SFM pr0n also. Porny VNs are sold on Steam, and you can see a lot of Daz faces in them these days.)

    Also, we were chatting about "companion" dolls, and then I saw that they sell animatronic dolls. (/surprised pikachu meme face)

    If one were into dolls, one could also do this as well with the doll's face, to make companion dolls even more "alive". (/another pikachu face)

    Amazing, the (socially distant) times we live in.

    How many goodly creatures are there here! How beauteous mankind is! O brave new world, That has such people in ’t!

  • MelissaGTMelissaGT Posts: 2,611

    What is the name of the app?

  • WonderlandWonderland Posts: 6,891

    I found a few in the App Store and tried downloading the one with the best reviews but it makes you take a selfie, not use an image from your phone. Which one did you use? 

  • WonderlandWonderland Posts: 6,891

    Wow, just found one, Avatarify, with which you can take any photo of a face, including a politician or celebrity(!) and have your expressions and visemes on your face control it realistically. So basically any teenager could record a politician’s voice, re-edit it so it says something different, then make it look like the person is saying that! OMG! Social media and YouTube is already so full of conspiracy fake news crap, this just makes a zillion times worse! 

  • emoryahlbergemoryahlberg Posts: 133

    I used Reface.

    You can use a selfie, but you can also upload an image from your phone.

    So I just emailed myself a render of my character and voilà! 

    Why'd I do it?

  • WonderlandWonderland Posts: 6,891
    edited August 2020
    emoryahlberg said:

    I used Reface.

    You can use a selfie, but you can also upload an image from your phone.

    So I just emailed myself a render of my character and voilà! 

    Why'd I do it?

    That’s incredible! That’s the one I downloaded but right away it asks to take a selfie and there’s no way to choose a photo. How do you bypass the selfie thing? 

    Never mind. I just denied permission to the camera and now I can choose a photo, but it wants me to pay to use pro.

    Post edited by Wonderland on
  • emoryahlbergemoryahlberg Posts: 133
    Wonderland said:

    That’s incredible! That’s the one I downloaded but right away it asks to take a selfie and there’s no way to choose a photo. How do you bypass the selfie thing? 

    Hehe. Thanks! I'm pretty tickled by this, I must admit. 

    To select a photo from my camera roll I pressed the face image in the upper right corner of the "Featured" GIFs, and then selected the "Photos" button.

    I supposed I should add that I paid for the premium version. I'm afraid I don't remember if that was one of the premium features.

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,230

    well they did it with Carrie Fisher after her passing and a few other actors, I have seen some awesome deepfakes on YouTube and know at least one app got banned because people were concerned so yes this will be a thing.

  • kenshaw011267kenshaw011267 Posts: 3,805

    Several years ago someone put Nicholas Cage in everything, everything he wasn't already in obviously. It wasnt all that convincing then but with the sorts of AI's we have now...

  • FSMCDesignsFSMCDesigns Posts: 12,755
    edited August 2020

    This reminds me of this, LOL

    jeffimage.jpg
    1125 x 609 - 76K
    Post edited by FSMCDesigns on
  • NathNath Posts: 2,809

    Does anyone know of something similar to reface that runs on a computer rather than a phone?

  • kenshaw011267kenshaw011267 Posts: 3,805

    Faceswap is the one I messed with back when I did:

    https://faceswap.dev/

    I'm sure there are others.

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,230

  • NathNath Posts: 2,809

    I'll have a look at them when I have time to play, thanks!

Sign In or Register to comment.