Animated character against video background / backdrop?
teknostorm
Posts: 163
My deadline is for monday and Carrara is busy rendering another sequence, so I'm asking this here without trying first myself, so that I can get to it immediately as soon as Carrara is free -
I need to put an animated Genesis 2 character over a stationary video (the video is of swimming pool with moving water, but the camera is stationary on a tripod).
What is the way to do it? I will use an image sequence for the video, and I need the video to cover the whole screen as it is (no perspective distortion etc).
Daz 3D is part of 
Connect
DAZ Productions, Inc.
224 S 200 W, Suite #250
Salt Lake City, UT 84101
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
I've never worked with Carrara, I'm a Daz user, but I think Carrara works the same way.
This is my normal workflow for similar projects:
1) Set your render settings and camera up in Carrara to match your live action footage.
*I normally import a still image from the live action video as a background to make this easier.
2) Animate your characters accordingly.
3) Turn the background image off, and render out your scene with an alpha channel by saving to .png or .tiff image sequence.
4) Using a Video Editing program that allows basic compositing(most do), import the live action video and your image sequence from Carrara. Drop the live action footage onto it's own track, with your Carrara animation on a seperate track directly above.
At this point, there are many compositing tricks you can use to better integrate the different media sources, but the animated character should now more or less look as if it's part of the live action video.
**If there are foreground objects in the live action video, your going to have to roto them out so your animation can walk behind them.
Again, I've never used Carrara, but this should work.
I hope this helps, post back and let me know how you make out.
Thanks that is a good idea, exporting as png sequence and compositing later. Will let you know how it goes!
In Carrara you may actually skip some steps and simply use the video footage in the Backdrop scene setting as a Map.
In this short video I've saved an avi animated render and applied effects to it in Howler and brought it back in as a backdrop and rendered V4 over it.
What's cool is that you can continue to layer the footage through to build your effects subtly if you wish. Like finish the whole thing off (or somewhere in the middle) with nothing more than a bit of fog rendered over the backdrop footage, etc.,
dinopt - it's working perfectly! I'm able to import the transparent pngs over my video files and composite!
Dartanbeck - That's a great video! However for me sequenced pngs worked better as in the video editor I can pan/zoom both the video and genesis character with some perspective difference to simulate camera movement, and also I can color match them as I wish.
Yes, but you can still use a video in the scene's backdrop and use the alpha channel to composite. Why? Well, if there's a specific action that you need a character to react to, you can scrub along the timeline to match it up. There's also the advantage of matching lighting and if you need to, Carrara also has a shadow catcher lighting model in the Texture room that you can apply to any object in the scene. Not only does it catch shadows, it also acts as a mask so that if you need a figure to walk and pass behind something, such as a post, wall, etc. Just insert a primitive or other simple shape, line it up with whatever it is you need to mask in the background plate and apply a shadow catcher.
This video uses a shadow catcher for the ground and animated grass.
https://www.youtube.com/watch?v=_8puXTRXt7Y
The shadow catcher won't catch reflections, but if you have a Pro version of Carrara, there are multiple render passes such as reflection that can be rendered out and used in a composite.
Awesome, glad to hear it's working out well!
This is some really good info, I had no idea you could import a video file into Carrara. I've held off on buying Carrara and have stuck with Daz Studio because I do so much post work in outside software.
I normally render everything out as layers regardless, even if my scene is 100% animation, so that I can apply post work individually onto each layer. But if importing a video for playback as a reference is possible in Carrara, I may have to reconsider purchasing it........
Yes, but you can still use a video in the scene's backdrop and use the alpha channel to composite. Why? Well, if there's a specific action that you need a character to react to, you can scrub along the timeline to match it up. There's also the advantage of matching lighting and if you need to, Carrara also has a shadow catcher lighting model in the Texture room that you can apply to any object in the scene. Not only does it catch shadows, it also acts as a mask so that if you need a figure to walk and pass behind something, such as a post, wall, etc. Just insert a primitive or other simple shape, line it up with whatever it is you need to mask in the background plate and apply a shadow catcher.
This video uses a shadow catcher for the ground and animated grass.
https://www.youtube.com/watch?v=_8puXTRXt7Y
The shadow catcher won't catch reflections, but if you have a Pro version of Carrara, there are multiple render passes such as reflection that can be rendered out and used in a composite.
You and Dartenbeck are doing a better job marketing Carrara than Daz, lol. I'm seriously reconsidering upgrading to Carrara.
If the masking feature works well enough and cuts down on roto-work, that would justify spending the money for Carrara.
This is some really good info, I had no idea you could import a video file into Carrara. I've held off on buying Carrara and have stuck with Daz Studio because I do so much post work in outside software.
I normally render everything out as layers regardless, even if my scene is 100% animation, so that I can apply post work individually onto each layer. But if importing a video for playback as a reference is possible in Carrara, I may have to reconsider purchasing it........
You can also use movie files and image sequences in shaders. In fact, almost any shader component in Carrara can be animated.
This video uses a movie file as a shader on the black hole object. I also rendered my scene in "layers" and composited them in my video editor.
https://www.youtube.com/watch?v=iS-sNje4k0o
You and Dartenbeck are doing a better job marketing Carrara than Daz, lol. I'm seriously reconsidering upgrading to Carrara.
If the masking feature works well enough and cuts down on roto-work, that would justify spending the money for Carrara.
I can't recall the user's forum handle, but Paul Scammel did a video test using couple Easter Island style head models placed in a garden and used SynthEyes (if I recall correctly) for camera tracking and a shadow catcher to render a scene he later composited into his background plate. The Carrara Demo reel from 2010 has his shot at about 1:09 in the video. There's also some other impressive work in there showing particles, physics, lighting, etc.
https://www.youtube.com/watch?v=Iw67NrdrDug
You said the magic word (phrase?) - camera tracking.
Rendering out image sequences and Camera Tracking outside of Daz is fairly straightforward, I've gotten great results with AE's & The Foundrys Trackers. The video linked to my signature is footage I shot on a Canon DSLR, animated characters rendered out of Daz, tracked and composited in AE. I'm fairly pleased with the results considering the lack of a match-moving plug-in inside Daz.
It would be great to be able to track live action footage and import the tracking data into Daz(or Carrara). Which is why I've held off on Carrara.
Still though, the masking feature you described earlier is probably worth the investment.
You said the magic word (phrase?) - camera tracking.
Rendering out image sequences and Camera Tracking outside of Daz is fairly straightforward, I've gotten great results with AE's & The Foundrys Trackers. The video linked to my signature is footage I shot on a Canon DSLR, animated characters rendered out of Daz, tracked and composited in AE. I'm fairly pleased with the results considering the lack of a match-moving plug-in inside Daz.
It would be great to be able to track live action footage and import the tracking data into Daz(or Carrara). Which is why I've held off on Carrara.
Still though, the masking feature you described earlier is probably worth the investment.
I'll be sure to check out the video tomorrow. You can also import camera tracking data as I recall. I think SynthEyes and maybe another program (can't remember the name) export to a carrara format or one that Carrara can use.
Yup, it's SynthEyes, Evilkoolaide mentioned he uses it and it has a Carrara exporter in a fairly recent thread:
http://www.daz3d.com/forums/discussion/53718/P15/#780469
These are some great infos and tricks! But I'm going to have to hold off trying these till I've finished with the deadline!
This is some really good info, I had no idea you could import a video file into Carrara. I've held off on buying Carrara and have stuck with Daz Studio because I do so much post work in outside software.
I normally render everything out as layers regardless, even if my scene is 100% animation, so that I can apply post work individually onto each layer. But if importing a video for playback as a reference is possible in Carrara, I may have to reconsider purchasing it........
Guess what?
We can even use video and/or image sequence files as textures! Yes... textures!
In that "Walking in the Storm" video in my link, I have added that 'rain' filter I've used in Howler on the scene onto the bump and highlight texture maps for V4 as well as the actual color maps, and then the same for the hair. They really kinda suck in that example except that it shows that it is possible! So then I tried to use video footage as a height map in the terrain editor - but that was a no-go. However, Carrara's Displacement channel in the shader room is so powerful that PhilW shows in his Advanced Training course how to build a whole city using nothing more than a shader on a flat plane primitive! So... yes... I could actually make moving height map terrains - just not using the terrain modeler.
Carrara never ceases to amaze me with how endless the possibilities are.
casual has a script for a free one https://sites.google.com/site/mcasualsdazscripts/mcjexportcameratopz3
I think his script only works with Daz 3 and lower, unfortunately.
At least that's what I read somewhere.
*Edit: Wow, I just checked the link you posted, and it looks like he updated it for Daz 4. How I missed out on this is beyond me.
Thanks for posting that link. I'm going download it and mess around with it later today.
I really like the idea that Carrara can use video files for textures and that we can layer renders over videos and just composite away as much as we need to.
Howler has a feature where you draw a path with your brush, then undo the result of the stroke. That stroke can now be replayed over any duration of an animation. Using an appropriate brush* we now have the ability to very simply draw, say, a drizzle of blood running out of the mouth down past the chin, etc., directly on a texture. Switch to the Spec map and stroke-play the appropriate brush to enhance the highlight for the drizzle, and then again on the bump map... simple, effective, and it all works together wonderfully. Using maps that already exist for the figure being used, we can fake a reflection of a TV that is turned on with a show playing on it onto the surface of eyes, animate whiskers disappearing as the character shaves, mud splatters, moving sweat or water or milk or whatever running down or across the character... it's just cool! And it can be a LOT faster to achieve the needed results than relying entirely on reflections in the scene, when only simple effect is actually required for the shot, rather than toiling over physics to get something to drip onto the figure, etc.,
Video files used can be either avi or image sequences, and the playback can be controlled using the timeline and tweeners, just like anything else. So oscillate can repeat an effect over and over with ease, for example.
Since texture maps can be easily used in so many important parts of a shader, we can apply these techniques to multitudes of ideas, like water puddling on the floor, a screen of a monitor changing, moving bumps with highlight designations which follow the bump moves perfectly as well as the color, perhaps even completely displace something enabling nearly any kind of effect. Being able to select Add, Subtract, Multiply, Divide, Mix, in so many shader channels - along with the fact that we can then use mixers or other forms of blending to actually change from one master shader to another in oh so many ways, makes the endeavor of using Carrara for nearly any kind of effect a very fascinating endeavor. Watching chapters of PhilW's (Infinite Skills Courses) "Advanced Carrara Techniques", and now "Realism Rendering" regarding making or tweaking shaders is really cool - same with Mike Moir (mmoir) and how he builds shaders right before your eyes, all helps to show how easy we can apply really advanced ideas to shaders - ones that I never thought to try before - which can be most beneficial things to keep in the recesses of the mind for future use. Phil, making a whole city shape out of a simple plane using a displacement shader had me thinking: "okay, I see where this is going" right from the start. But then he does another thing that I didn't expect... and perhaps another, and another... then he actually works in the rest of the same shader to give windows, roofs, and such to all of the buildings... whether we actually use this information to make a city is not relevant. The fact that we now know how to work with layer lists, wires, squares and such to so many varying degrees is powerful.
Hey @Dartanbeck -
Just happened across this thread. Could you explain this line further: "Since texture maps can be easily used in so many important parts of a shader, we can apply these techniques to multitudes of ideas, like water puddling on the floor"
Would like to add that ability to my toolset if you could explain in more detail.
Thanks
HI Mosk
in a shader,. you'll normaly use texture maps,. colour and greyscale, they'll be used in different channels of he shader to produce different results.
In carrara,. you can use an Image,. or an Image sequence,. or even a video clip.
if you have a clip of video of "anything" you can use that in any texture slot in any channel of a shader
whether that's a clip of real footage, or an animation
if you created a simple animation (greyscale) of drips growing into puddles (small white points becoming larger areas)
you could then use that animation in the bump / displacement / shininess / reflection channels of a shader,.
you'd get an animated effect.
If you had a little clip of some electrical effect,. you could add that to the Glow channel of a figure's face or the displacement or bump to create a strange effect
you could also have an animation of changing skin texture,. or drive the transition between textures in a mixer, by using an animated blender ,...or whatever effect you're after...
slightly off topic,..
Over at Videocopilot recently, there was a really nice example tutorial posted (Realistic raindrop effect) of using a simple animation (greyscale) made in AE,. to drive the displacement effect to create realistic raindrops and run's
hope it helps