"Why it’s so hard to make CGI skin look real" - VOX
Steve K
Posts: 3,241
The video mentioned is here:
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
imo primary factor is in the lighting. too much light you lose the details, ambient overexposure.
real skin is flawed.
Alita, bless her.
Yet, for real people, look at Hi Def TV and notice that actors (nevermind actors... I see news presenters) are being made up to look 'poreless' with perfect spraytans and fillers! Kardashian-esque.
On CNN or FOX in the US, the amount of makeup is so over the top along with blinding white teeth vs when I compare it to British TV where a more natural look is 'permitted'. Covid even made our news people look more normal as during the height of it all they had to do their own makeup... at least on BBC. It was easier on the eye to be honest. I could see real skin and crow's feet etc.
that's a *really* good observation/point...
--ms
I think the answer is between Jurassic Park and Lord of the Rings glorious visual effects VS Jurassic World and Hobbit fake and plastic CGI.
Lighting and context have a huge part on this. You can have 1 billion polygons and the best shaders ever, but our brain keep telling you is fake because it have impossible lightings and impossible physics.
CGI artists are worried about let you know how perfect and detailed their 3D model is, so they keep their model in perfect light positions. But real life is not like that, you can't see full detail and percet lights for everything all the time. So, i think is like the first Matrix designed to be a paradise for everyone... we humans just don't buy it. We need imperfection to buy something as real.
That's a very cool feature.
This one is eight years old already but helps to explain a bit deeper into how Alita turned out so much more 'real' than many other CG attempts. While the video in the original post addresses the skin, Weta Digital's "Tissue" is what drives its final shape and characteristics, which takes it to levels I doubt I'll ever really go for for my fun fantasy work - but one never knows.
Quote from video: "See? I even put my monitors to sleep just saying that" LOL
Minor point for storytellers - willing suspension of disbelief contains the word willing.
I think the audience for Gerry Anderson's puppet shows could tell the actors were not real. The audience willingly stops looking for skin flaws when (something) happens. Find out what (something) is; it might be a lot cheaper than obsessing with subsurface scattering, vein distribution, and the number and size of skin pores.
If your goal is to make even a skeptic think it is real, that is different, of course.
Even if you are dealing with a skeptical audience, the best strategy might still be something other than actually making the object as detailed as possible. The brain is trained to fill in blanks when (something) happens. Learn what (something) is.
See - https://www.carnivalofillusion.com/best-magicians-of-all-time.php
A stroll through the Show Your Renders thread will reveal that Magaremoto has been working on this for a long time - Micro-displacement, and he's been doing it in Carrara making his own sophisticated, highly detailed displacement (rather, Micro-displacement) maps to help achieve those higher standards of visual appeal - realism. Just look for the spooky, lifeless 3D Scan corpses and read his notes.
Jonstark has shown some remarkable CG skin renders recently. He's mentioning liking the Carrara engine but also mentioning Octane, so I'm not sure which was used for these images, but they look great!
I find that, like Jonstark says in that post, Octane and Iray certainly make it Easier to get these highly realistic results, but that is likely due to more modern material options and conversions being set up with a formula that just works well for achieving good results, and then the better the maps we feed into it, the Even More realistic things start to look - and of course lighting is what truly sets it all off.
Carrara is absolutely capable, in fact I think that's what Jonstark used for this and, like he said, PhilW's Realism Rendering course has some invaluable workflows shown to help achieve the more modern, high-end look of CG realism.
And this all comes back around to what Magaremoto has been doing, getting a much higher level of detail into the skin.
In some of his posts on the subject he explains some of the procedures he was using to create his micro-displacement maps to enhance the surface.
And then comes Subsurface Scattering - something that I've still not found a rewarding solution to in Carrara, so I attempted to fake it in the translucency channel. But after working with Iray and Octane for a little while, I may just try SSS again in Carrara - thinking that I've been approaching it all wrong before.
I never really wanted to be ultra-realistic. In fact I like the sort of painterly look I get from my characters. But the more times we produce something with a bit more realism than normal, the more we want to improve that and keep pushing the limits - even if we don't realize we're wanting it!
This one is all Iray, which actually looked more 'real' before I stylized it in Affinity Photo
Five minutes of Alita being cute. A great little glimpse of how spectacular she is if you haven't seen the movie, and a great way to focus on her realism if you have
Combined with perfect facial symetry, the pore-less skin just glares 'FAKE' at ya dow there in Uncanny Valley. It's why I love Zbrush... I can muck about with facial features without removing polys for some 'super-asymetry'. As long as I am very careful!
Alita is just so cute... too cute to be a Battle Angel!
...but it's her cuteness that makes her such a Badass!!!
Less detail, not more. Your audience has a brain, MISUSE it.
A few years back, the stage magician Penn Jillette of Penn and Teller fame was hired as a consultant by CG arts production companies. One of the articles I read at the time was scathing. As a consultant, he did not hide his disdain for the hyper reality focused, although he eagerly embraced and used all the technology he could find. To the dismay of the CG technicians, he told them that their focus on getting the details correct was all wrong. Magicians had been 'tricking' the UNWILLING suspenders of disbelief for at least two millenia, and the CG computer people should spend a little more time understanding how.
Since then I have been more and more interested in two divergent productions.
1) Return to the completely unrealistic 'claymation' models that drew me to storytelling
2) Embracing the 'realistic' trend that others enjoy, and follow the issues that Penn Jillette raised as a consultant.
I am sure I have posted a link to one of his critiques before, so I will try to hunt it down again. In the meantime, enjoy a couple of reviews of the VR game that Penn and Teller came up with.
https://www.vrfocus.com/2019/06/preview-penn-and-teller-vr-frankly-unfair-unkind-unnecessary-underhanded/
https://thenextweb.com/news/review-penn-tellers-vr-experience-is-hilarious-magical-and-decidedly-cruel
Edit to add a caution
https://deceptionbydesign.com/wp-content/uploads/2020/08/Shift_02_Henderson-Anybody-But-a-Magician.pdf
Realistic skin materials can do nothing while the joints are looking like rubber hoses, and details mean nothing when armpits are cut to the chin, like just V4 has. What really means to me is the whole visual impact where reality is all perfect silhouettes with fine tuned limbs movements. And, of course, soft shadows, even in cartoon execution.
Wow - Yeah! I understand that's why Weta Digital's "Tissue" (Video above) was such a major breakthrough. They actually used MRI to see what the joints are doing in certain poses (Relaxed to extreme, I'd bet) as well as how the various muscles move, twist and change shape - how stiff they are at different poses, etc.,
So then the muscles and fat tissue become a bit of a flow and constraint properly compared to how the joints actually separate and contract, all creating the shape of the skin outside.
Since it was/is all done according to real data, their results are spectacular.
Many years later, they went to microscopic details on the skin, poured real science into the sub-surface, and even more attention to what are called micro-expressions, which are involuntary shapes that each person makes without ever realizing it - and no two people do the same micro-expressions.
So when they made Alita, Rosa's (Actress) true performance comes through better than any attempt prior. At least as far as the general population is aware of. I absolutley love how well they nailed the scene where she eats an orange for the first time - just spectacular, and gives beautiful demontrations on how micro-expressions help to humanize a bunch of pixels!
Took up the challenge, my general skin shader so far... no textures just noise and colour. Need to find a way to make freckles.
Still looks a little chalky imo, also need to tweak the sss.
mschack, the 'spots' shader under 'natural patterns' is good for skin blemishes, pimples, and I think would be good for freckles too. Use an operator: Mixer between the main color and a very slightly darker color that's similar, with the spots function in the 'blender' channel for the mixer. You can do spots within spots within spots with multiple mixers inside of each other. Take the 'spot size' down so that it's small spots, then take the 'Blending' waay up to 95% or more to make sure they aren't dominating the surface area. Copy/paste the spots setting into a mixer for the bump and specular channels too, and you can make it so the spots seem to be on the surface of the skin or protruding. No specular and no bump will make it seem like the spots are blemishes just beneath the surface of the skin. At least that's my theory at the moment; I could be wrong :)
Ironically I've just been finishing up on my own try at a procedural skin shader. It's just about there and ready to share, hopefully I'll be able to put up something on sharecg in the next couple of days, along with a post explaining my theory/method (which mostly consists of ripping off better cg artists that came before me lol). :)
I hadn't really considered freckles before though... so after I saw your post I had to test them. I was so focused on subdermal skin blemishes I didn't even think about freckles, but freckles are important to selling realism I think. The limiting factor is that they don't look natural if they are consistantly and evenly applied across the skin. Most people get a very few freckles on the foreheads, with a bigger splash across the nose and onto the cheeks of either side. After pondering for a bit, I decided to 'draw' a rectangular layer over my main layer, using a red-ish freckle color and a mixer in the opacity channel between 0% and the color, and a spots shader as the blender channel. Actually it was a mixer inside of a mixer, with Anything Goos controlling the falloff (that's why there aren't any freckles around the eyes or right around the nose holes and lips, as those are the edges of the shading domain. However while it's a decent result, it's also too 'square'. I think maybe it would be better to do multiple of these layers using the Oval layer tool instead, to get even more realistic results (an oval over each eye and cheek, one over the nose, one over the forehead, maybe a little one on the chin).
Oh yeah, editing to add: The eyebrows and eyelashes are Carrara hair. I think they came out pretty decent, what do you guys think?
Thanks Jonstark I will try that, will have to mix one of my mixers lol. Your shader looks good!
So far I like the detail in the body my shader is producing but the face still looks too clean. Perhaps i will have to make a seperate shader for the face.
Not that i use figures all that often, but i guess G2 has no belly button. Maybe there is a slider lol...
Couldn't resist tinkering and experimenting a bit more on the freckles. Left the rectangular layer but took down its opacity 50%, then added several different oval layers (one on each cheek, one across the bridge of the nose, one on the forehead just above/between the eyebrows, one on the chin) with varying spot intensities. Seeing a lot of interesting possibilities in this layer approach; looks like it's very possible to add specific facial details to various areas of the face with layers while still keeping the whole thing procedural.
quick 'from the hip' observation on freckles, inspired by your experiments:
to my eye, realistic freckles seem to be a mix of spots that are anti-aliased against their base skin, with the larger spots being even more blended/transparent to that skin. Not quite sub-surface, but almost a watercolor melt into the skin. maybe 60% / 40% - sharder spots to softer larger spots, with the larger spots melding more.
An uncanny-valley effect creeps in when the spots look painted-on, so any form of transparency and edge/blend in that layer that softens the freckle edges at various levels would counter that effect. I can't recommend an approach with any expertise/confidence.
This is a really intriguing path for skin in Carrara, as your earlier posts of that blond woman are probably the first set of 'realistic' skins that stand up to today's standards - and I've got most of the available skin-sets/shaders available to us, and ... they're not wrong/bad, but your posts just popped for me. (like "wow, that's Carrara?").
In this last set, the 'chalky' effect is *really* improved by that last change where there's some oily shine/reflection/specular - which apparently makes a huge difference.
Watching your progress like a hawk! (no pressure!)
best,
--ms
(w minimal edits to strengthen the ideas)
With a hint of freckles
interesting update. with this test, two observations/thoughts:
- I'd be curious how a bit more of the mentioned oily shine would affect the result of your figure. a matte/pore-like tooth to the skin would probably add a bit of that 'something' to reduce the chalky effect you mentioned too.
- it seems like softening the freckle edges reduces the available contrast/impact of the starting freckle image, which makes sense. We may have to simply make the freckles larger (a bit darker too?) to maintain their presence when blending/anti-aliasing them into the surface. You used the word 'hint' when describing the freckles in your render, so the effect you've created may be intentional. I'd be curious how strong you could go before the effect becomes a problem on the other end of the spectrum.
good stuff,
--ms
I went to check out a few reference pics of people with freckles (probably should have done that beforehand, lol) and realized my freckles were just a bit too red. Kinda looked a bit like bloodspatter. Needed to be more brown.
Also, I realized that most people's freckles are much bigger than the little dots I was doing. So I increased the size of the spots.
In addition, I lowered the opacity of the various Oval layers of freckles to between 15 - 30%, really trying to make it 'blend in' naturally. Can't think of a way to soften the edges of each 'freckle spot' however.
Then, for kicks, drew some ovals around the eyes and voila: now we can do basic procedural makeup too (look ma, no maps!) :) Gave her a basic purple eyeshadow to test.
Much less successful in drawing some oval layers to put on some blush, however. The edges of the oval need softening so they blend in, but offhand I can't think how to do it. Anything Goos doesn't work, because there's no real shader edges nearby for it to use to calculate where the falloff should begin... So when I tried to put on blush, I just ended up with hard circles. Softening it by lowering the opacity didn't do the trick either, as it lowers the opacity uniformly. There's probably a shader that can do this, but I'm blanking on how...
Eh, threw on a quick hair and added a layer of lipstick. This probably doesn't help as far as testing, but I just wanted to see...
To me, the procedural attempts at making human skin very much fail if realism is the target. They might make for a decent anime/toon look, but nothing beats a good set of texture maps for making CG characters look more real.
But that's just what I see.
My youngest sister is the most brilliant artist I've ever met and, to her, the tiniest measurement of the skins surface has countless shades and hues. After she told me that I looked really closely in the mirror and she's absolutely right.
Indeed, these multitudes of change get smoothed out in our normal perception as more of a base color, but trying to reproduce human skin without incredible amounts of change in the tiniest of scales results in something that simply does not look real.
Definitely not trying to judge, nor to discourage to continuing trials and attempts, just making an observation.
This thread caused me to replace my Carrara version of Rosie 5.2's skin maps with the originals, which are 4,000px. I normally reduce maps to 25% their original size because I've always been more interested in speed. I always keep the original files in a folder where they normally reside, so I always have access to them if I need the extra detail.
Yep. Makes a big difference. Skin pores show much more accurately and she just looks a lot more real. After seeing that, I put my reduced maps back. To me, even those lower resolution skin maps still look better than procedural - even though I'm not really going for realism.
Diomede's points are awesome as well. Instead of going for Realism Believability, some filmmakers instead opt for Audience Acceptance.
Pixar does this well by taking actual realism out of the equation, while still using SSS, soft shadows and minute-scale expressives that warm (or chill) the heart of the viewer.
I tried an experiment a while back - making my characters look drawn or hand painted. I had a lot of fun with that but it was starting to pull me too far away from what I was actually trying to accomplish.
Oh yeah Dart, I completely agree. The more I dabble in procedural skin, the more I'm convinced it's just not a viable solution for photorealistic results, texture maps just basically blow away procedural every time (even very poor texture maps).
After seeing that drawing oval and rectangular layers on the shader is viable for 'drawing' separate patches of skin that has different looks (I only did it for the freckles, but it proves that it is possible to do), in theory it might be possible to make a realistic skin texture that could stand up to texture map realism, but it would be incredibly time consuming/difficult to do. A few problems would have to be solved:
- First, we'd need to figure out a way to control the falloff of the effect, so that one drawn on layer meshes seamlessly with the neighboring region. Anything Goos would have been my first guess for this, but it won't work sadly, since these 'extra layers' I'm 'drawing on' aren't read as real shaders by Anything Goos, it's reading the edges of the shader as being the whole Face shader, not these little ovals and rectangles I'm adding on top. In order to make pores look natural, they would need to 'stretch' in various directions in various places on the face (like under the eyes for example) so that when we zoom in it doesn't look like a perfectly even cellular pattern, but actually looks like it grew naturally with stretches and squishes here and there, like real human skin. You can draw an oval layer and stretch the 'cells' (color/highlight/bump), but if you don't have a way for it to seamlessly merge at the edges with the skin in the next section, we're just going to see lots of ovals all over the face, lol, as we perceive the boundary line. I can mostly get away with it with freckles because they're just spots within the ovals, so it's harder to percieve the boundaries of the ovals.
- The next problem is how difficult it would be to do, you'd need someone that is practically a genius in observation of realistic human skin, zooming in big time to draw tons of tiny ovals and rectangles all over the face (and other skin parts if you're going to do the whole body). What I did with the freckles was only a total of five ovals and two rectangles, but it still took some time. The drawing tools onto the model aren't very responsive, and Carrara isn't fast in using them, nor are they precise. In every case I would draw a rough circle or rectangle, but it wasn't in the right place, then it was a matter of fine tooning the shader itself to move it to the right location I wanted, each update in the texture room took some time to calculate, then I would see it wasn't in the right place, change the numerical position a little bit, wait for update, repeat. And while I didn't have any noticeable problem, I'm guessing that if someone was adding hundreds of little ovals/rectangles, that after a while the more and more drawn on extra layers are going to add up and make Carrara even more sluggish to respond and more prone to crash. And the more math it makes Carrara do, I'm betting that after a while this would slow render speeds too. Though I have to admit I didn't hit any problems, but then I didn't add all that many drawn-on extra layers.
- the last problem is being bound to the limitation of using only ovals and rectangles. Supposedly there's also the option to draw a polygon layer, and if true then we could get detailed right down to each particlar polygon, or selection of polygons, but for the life of me I can't figure out how to use that function or make it work.
Anyway, my conclusion is even if it's theoretically possible to do, there's no reason to believe a random genius would show up and spend the huge amount of time to actually do such a thing, nor that it would actually be a useable result. :)
That said.... is there any use for a Procedural human skin shader? Answer... I'm not sure, but I think there might be.
The main use (I would guess) would be for background people in a scene, not the stars of the show who are in the spotlight. Although computers are getting better and better, using procedural means no texture maps, which as I understand it would relieve a large portion of the memory requirements for a scene. A huge crowd of people in the stands of a stadium, a bunch of people walking through a mall or subway, etc. Those kinds of situations seem like they would be good for using procedural skin, if the people are close enough to see clearly but not in closeup where we can see the problems.
However, that said I'm not sure. These procedural skin shaders I'm dabbling with do use SSS, and while it's a tiny amount of time calculating, it could add up in an animation, making it still unfeasable to use (though I guess it would be fine in still shots). Of course, the SSS could be turned off, and probably wouldn't matter much for background people anyway, so maybe this approach could still be useful..
A PBR skin set needs a good skin roughness map.
On a tangent note, some movies have particularly good face closeups. I recently saw the 1999 movie "Election" starring Matthew Broderick and Reese Witherspoon (7.2 IMDB, 3.5 stars Ebert) because Netflix added it. Broderick is a highschool teacher up against an insurmountable force in student Witherspoon. At one point he gets a bee sting on his eye, a comic genius move IMHO. I'm not a big Broderick fan, but some of the many closeups after that are hilarious. He plays a great schlub, so maybe I am a fan.
Broderick for me is immortalized in Ladyhawke, just a superb performance. He was pretty good in Ferris Bueller too, but it doesn't hold nearly the place in my heart that Ladyhawke does. So I forgive him all his later work (not all of which was bad, but none of it ranks among my favs)