Compositing and Post Work - What is it, and why should I care ?
JoeMamma2000
Posts: 2,615
Well, after getting all kinds of grief for sarcastically implying that postwork is “cheating”, and that some folks here consider postwork to be “blasphemy”, I will now perform penance and try to atone for such a horrific act of sarcasm.
And since I was proven very wrong in assuming that most people here are not really too interested in compositing and postwork, I will attempt to take advantage of this widespread interest and run through the basics of what compositing and post production work are, and why people might find them useful.
Hopefully this will help some who haven't yet delved into the world of post production work. Since I don't have a lot of time to spend on making this complete right now, I'll add to it in this thread as I find time.
And I’ll try to do it without smiley’s, since apparently those too cause some to get greatly offended.
So, what are compositing and post-production work? And why might you need it?
Well, compositing and post production are extremely common and important aspects of virtually all professional audio and video production.
As an example in the audio industry, think of your favourite musical group or singer. It’s likely that when they go to a studio to record their latest hit, instead of all performing together with one microphone, they record the separate band members in separate audio “tracks”. They might record the drummer by himself, and he’s probably banging away in a soundproof room, and listening with headphones to the music produced by the other band members.
Same thing with the lead singer. He/she might even be in a different city, singing alone in a soundproof room. And at some point, all of those separately recorded tracks are brought together into one recording. It’s called “compositing”. Bringing separate parts together into a whole. Here’s an image of one of the industry standard audio compositing software tools, Avid Pro Tools.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
The same compositing concept applies to visual media, like feature films or music videos or TV commercials. You do things in parts, and then merge them together for a final product.
Why do they do it that way? Well, for many reasons. Here’s an illustration that may help to explain.
Let’s say you are contacted by someone who knows that you do 3D stuff. And this producer says “I want to make a short, 15 minute film, starring a world famous superstar named Angelina….and I want you to make the film showing Angelina walking thru a small town in Wisconsin, and suddenly she gets attacked by an army of droids from Mars.”
So you say “Of course I can do that…piece of cake”.
Then, after weeks of negotiations with Angelina’s agents and lawyers, you settle on a contract where she agrees to shoot on location in the small town in Wisconsin for a price of $1.6 gazillion. Then you spend a few more months arranging to bring a 50 person crew to this small town, including a camera crew, sound crew, caterers, portable dressing rooms, transportation people, admin people working on permits with local government, personal assistants, lighting crew, and on and on.
And then you go into the small town where you’ll be shooting and find out that there is only one really cool Martian droid available from a recent Martian attack, so you’ll have to do some fancy CG work in Carrara afterwards to add the army of droids. Piece of cake, right?
Now after you’ve pre-planned exactly what you’re going to shoot, obtained all permits and contracts, hired and transported all the people to the small town in Wisconsin at HUGE expense, you finally spend a few days filming her walking thru town, past the local pharmacy, and reacting like she’s suddenly in the midst of a terrifying attack of Martian droids.
And when you’re done, you wrap everything up, the crews pack up and leave for home, and the next day you proudly send the video files to the producer and show him what you’ve done.
And the producer’s response? “Ummm…I was thinking about it last night, and wouldn’t it be better if we could shoot it at night to give it some more drama. And I don’t like the background. Can’t we show her in front of WalMart or something to make it more realistic? And by the way, did you notice that there’s a dog walking thru the shot when the droids are supposed to be attacking?”
And your response? “Oh, poo…”
So what do you do? Well, of course, you call up Angelina’s people and ask if she can fly back to Wisconsin so you can re-shoot, right?
Well, no. The cost to bring back the same crew and the star and do it all over again is prohibitive to say the least. It just ain’t gonna happen, for many reasons.
Now you’ve got a multi-gazillion $$ star in high definition video, but with the wrong background, the wrong lighting, and a dog ruining the scene. And, after you closely review the footage you shot and talk to some other folks to get their input, you notice a bunch of other stuff you wish you could change.
Double poo…
Well, you should have planned for stuff like that, and done it right the first time.
Rule #1 with stuff like this: “When you do expensive and difficult stuff, once is enough”.
So what are the lessons to be learned?
Well, you should always plan for, and assume, there will be additions, deletions, and modifications to whatever you produce. And that applies to every single aspect of what you produce. There’s always a long chain of approvals required for any significant production. There are supervisors, directors, producers, clients, etc. And it’s guaranteed that people will change their minds as the production progresses. Guaranteed.
So how do you plan for changes? Well, you break your production into separate parts so you can individually add, delete, or modify any of them, and then bring the final parts together for your final production. You “Composite” them.
In this example, the footage your shot of Angelina isn’t going to be re-done. The cost to do that is beyond prohibitive. So you should have isolated and separated her footage in the beginning so you can change her environment, while keeping the core of that incredibly expensive and valuable footage unchanged.
And that is the key to understanding Compositing and Post-Production work: ISOLATION.
You ISOLATE every aspect of your production so you can add, delete, and modify if the need arises. You isolate your actors. You isolate your back ground. You isolate your visual effects. You isolate shadows. You isolate reflections. You isolate each 3D object. You isolate your lights. The list is endless.
And there are many, many techniques for isolating the parts of your production, especially when you are working in 3D. But the concept of Compositing and Post Production is all about isolation and modification/addition/deletion.
So how does the smart producer handle the shoot with Angelina in Wisconsin?
Well, you isolate. First, you isolate Angelina. How do you do that? Well, there is a technique called “chroma key” where you shoot your actor in front of a background that is a solid color (a particular color which is rarely found in real life stuff, especially humans) so you can easily erase (or “key”) the background later, leaving just the isolated actor. BTW, that color is often a pure, bright green or blue, which is relatively easy to erase later in post production. It’s easy to tell the software to delete every pixel that has an RGB value of (0,255,0), for example, without erasing other important parts of your image, since very few surfaces in real life are (0,255,0).
Once you do that, you can now place your actor’s footage in front of any other background, and have total control of how that footage appears. And you can later adjust any aspect of that footage.
And, BTW, since you only need your actor to do her acting in front of a green screen in a studio, you can save a TON of money by not requiring your actor (and assistants, support personnel, etc.) to make the long trip to Wisconsin. Just do it in your production studio, and add a background and whatever else you want later on.
Now, that all sounds good if you’re working on a multi-million dollar feature film, but is it really necessary for the average 3D artist? Sure, it can save you a lot of money in a big project, and makes it a lot easier to make changes when your boss changes his mind, but why should I care if I’m a one-man-band doing 3D for fun?
Well, if you care about the quality of your final product, and are interested in getting great results that you may be unable to accomplish with a straight 3D render (and do it much quicker), then maybe compositing and postwork are something you might want to consider.
There are many, many examples of how you can benefit from composting in your 3D work. For example, if there is an effect you want, but your 3D application can’t do it well, or takes ages to render compared to other apps out there. Simply isolate and composite that effect into your render.
Below is a super simple example of a composite that was integrated from multiple sources, to make up for, IMO, a deficiency in Carrara’s ability to render a believable fire/smoke simulation.
http://youtu.be/1pJ-8a5v1iU
The flames were rendered in a professional app called “FumeFX”. The smoke was generated by the particle system in a compositing app called “Fusion”. And the background was a simple render done in Carrara (or maybe Blender, I forget). The separate elements were generated separately, then brought together in Fusion for compositing. The flames had to be scaled and moved, and a Glow was added. Same with the smoke simulator. And the speed of both the flame and smoke simulators was modified so that they match, and look more believable.
Now, these are just the beginning steps of the comp, and more work needs to be done to integrate the flames into the background scene (interaction with the background, such as light and shadow on the walls and floor). But the point is that you can generate a higher quality product, with greater efficiency and a lot less work, if you become familiar with the concepts and techniques for compositing.
There is not always a green background on the pitures, in AE, there are several tools to do that on a real sight of which the roto-brush.
There are also tools to make correspond lightings, your composition is not coherent on this subject.
Now that we understand what it is, and have a little idea how it can help us, the question is, "How do you do it in your 3D app?"
Well, we mentioned that the KEY to compositing and postwork is ISOLATION. You need to find ways to isolate every component of your renders.
Fortunately, there are many many tools available to do that. And one of the key tools available in Carrara and virtually every other 3D app out there is called "Render Passes".
What are "Render Passes"? Well, you need some way to isolate every component of your rendered images so that you can separately add, delete, and/or modify that component. So, for example, if you wanted to isolate each rendered object in your image, you need some way to do that. Or if you wanted to isolate the distance infomation in your image, so you could apply a depth of field blur. What a render pass does is it gives you a separate layer or channel of your rendered image, and makes it an integral part of the image. And that separate layer or channel gives you a grayscale image, and the RGB value of each pixel of that image translates to, in the case of a depth pass, a relative distance from the camera to that point on the object.
And that, in a nutshell, describes what render passes do. They give you a separate image (layer or channel) that describes, using RGB values, the information you need to isolate that particular aspect of your image.
Another example: if you need to isolate an object in your render, the Object Index pass gives you a grayscale image where the grayscale value of each pixel is the same for each object. So for example, if you have a simple cube in your scene, the grayscale values for every point on that cube are indentical. That makes it very easy for you to select that object with one click: just select all RGB values in the image that are (1,1,1) for example.
And let's say you want to isolate only the reflections in your image, so you can blur them quickly rather than relying on the dreafully long render times for blurred reflections in your 3D app. Well, you can get a separate render pass that includes ONLY the reflections in the image. So you just take those reflections, blur them with a built-in blur function in your compositing app, and composite that with your main image. Voila, you now have nice blurred reflections that were done in a fraction of the time, and you had ultimate control over the results.
Attached is an image of an "Object Index" pass, which shows that each object has it's own distinct grayscale value, making it very easy to select just that object. However, normally the difference in grayscale values between objects is very small (presumably to allow for many objects in a scene) so you may have to do some tweaking to figure out exactly what value is which object. Here I tweaked the levels in PS in order to more clearly see the grayscales per object.
And here is an image for the depth pass from a render. In this case, the darker the RGB value of the grayscale depth image, the closer it is to the camera.
What you can do with that pass is to use it in your compositing application, which will directly take a depth pass channel and use it to calculate a Depth of Field blur.
Now here's a further example of how a render pass such as "Object Index" can help you do things quicker and better.
The first image is a straight render from Carrara. Now let's assume we don't really like the rear wall for some reason.
Well, if we rendered with an "Object Index" pass, we bring the image into our favorite compositing app, select the Object Index channel, and select the rear wall object (using a low "tolerance" in your selection). With one click just the wall is selected. Hit "delete", and now you have a transparent matte where the wall once was.
Now you can insert a new layer with a different wall image, and make any other additions or changes to it you'd like.
Now let's assume I want to boost the reflections in the car body. I render a "Reflections" pass, and it comes in as a separate layer in my image.
So I combine it with the original image layer in such a way as to enhance the reflections. In this case I cut down the Opacity of the reflections layer to around 25%, and used a "Linear Dodge" additive blend for the Reflection layer.
Not sure I'd do this in real life, but it serves as a nice example of just one of the many millions of ways you can modify every aspect of your images.
Thanks for the detailed description and examples.
Now that Fusion 7 is free for all and not just a tool for those with high budgets, any suggestions on where to start learning the Fusion basics for a Carrara user?
Since it is a "pro" tool, does it have a steep learning curve?
https://www.blackmagicdesign.com/uk/products/fusion
Thanks for the screenshots and the explanations. Appreciate it.
Fantastic Joe. Your explanations will help many in this forum.
If you want to learn compositing, I STRONGLY suggest you don't start by learning the software. There are good books out there you can download if you want to understand compositing and postwork. Fusion is just a tool, and it won't teach you how to composite.
I would recommend a book like the Ron Brinkman compositing book. It explains the basics of what you need to know to understand images and how to manipulate them.
And, unfortunately, since Fusion's history is as a professional tool, there aren't many tutorials and videos from users out there. Yeah, there are some, and the developer has some too, but those generally assume a knowledge of the basics.
By the way, one of the most used and basic tools in compositing is a color correcting tool. Fusion has it, all compositing apps have it.
And I just searched the Fusion manual, and there is not ONE explanation of what it is and why you'd need it and where you'd use it. Nothing.
Now, if you start using Fusion, or any other compositing app, and don't yet know the basics about what all this terminology means and how you'd use it, you can spend years playing with it and never know what to do with it. Compositing apps are filled with features like this, and you're far better off learning the basics first before you play with the software.
On the other hand, if you realize that, for example, much of what you see outside on a clear, sunny day has a slight blue color cast due to the diffuse bounce light from the atmosphere which illuminates everything, then you'll start to realize that "hey, maybe if I'm compositing something that was shot in a studio, or rendered in a 3D app, onto a background shot outside, I'll probably want to give the indoor elements a slight blue cast using a color corrector". The software will never teach you stuff like that.
Another example that is pertinent to the discussion about Object Index pass....
If you had never learned what an Object Index render pass is, and how you'd use it, you might look at the Fusion manual and read this: "Any tool in Fusion can use the Object ID and Material ID auxiliary channels to generate a matte or a mask. The settings used to accomplish this are found in the Common Controls..."
And you'd scratch your head, not sure what an Object ID is, or what a matte or a mask is, and discard the info and move on.
On the other hand, now that you understand what an Object ID is, you realize "WOW !!! That's awesome !!! I just render an Object Index pass, bring the image into Fusion, and instantly it will generate mattes for each object so I can quickly isolate the objects without doing any work !!"
Now that we understand how the key to compositing is ISOLATION, we can start to look around in our images and think in terms of "Hmmm....if I could isolate this particular aspect of my image, and change it to make it better, what kind of stuff would I want to change?"
Well, in 3D you can isolate just about everything. It's really only limited by your imagination. And that's one aspect of compositing which requires a different mindset. You need to start thinking in terms of making your images better, one component at a time.
Now, in the last example I decided I wanted to change a background wall, and insert a different wall with a different, and more interesting texture.
Well, I said you can isolate just about anything in your image, so why don't we isolate the UV information for that wall and just re-texture it in your 2D compositing app? No need to re-render. And you can quickly just zip through different textures in your compositing app and see which one looks best.
HUH? Are you serious?
Yeah, you can do that. Now, keep in mind that unfortunately Carrara is extremely bare bones in terms of compositing features, but luckily there are much better apps out there that will help you do stuff like that with your Carrara renders.
So if you look thru the list of available render passes in Carrara, you'll see one named "Surface Coordinates (UV)". And if I render the same scene with a UV pass I get the following image as a separate layer for the U coordinate information. Something similar is generated for the V information.
Now, you can take that information into, say, Fusion, or AE, or Photoshop and use it to re-texture just that rear wall. You'll also need to isolate that wall with an Object ID pass, and unfortunately like I say Carrara's compositing features are pretty much non-existent, so you'll need to do some extra work, but it's still a neat way to do re-texturing in certain situations.
If nothing else it gets you thinking about how to isolate different aspects of your image so you can change them later.
Previously there was a discussion here about "faking GI". And one of the techniques that is often used is to insert lights in the scene which mimic the actual direct and bounce lights which are present in a real environment. You vary the quantity, the intensity, the color, and often the shadows which are cast, until you have what looks to be a believable scene.
Now, post production and compositing can help with "faking GI" in many, many ways. You can access and isolate ambient occlusion, shadows, and tons of other aspects of your image.
But one area that was mentioned was specular highlights.
One reason you might want to simulate diffuse bounce light with a "wall" of low intensity lights is so you don't have a harsh specular highlight that might occur with a single, bright light. If you have a single bright light simulating the diffuse blue sky light coming in a window, for example, it will tend to generate a harsh specular reflection on opposite walls in the room. So one way to alleviate that is to use a wall of very low intensity lights which won't generate such a harsh highlight.
Another way to alleviate that issue, however, and which might prove a more efficient and flexible solution is to (yeah, you guessed it...) ISOLATE the specular reflections in your image and modify them. And while you're at it you can also modify shadows using a shadow pass. Make them softer using a Blur tool. Or you can add a blue cast to simulate the blue sky light.
So not only can you improve your images, you can do it with total flexibility, and you can do it incredibly quickly, without long re-renders to check to see if it looks good, then another re-render, then check again.
Doing things in 2D can be far easier and more efficient.
Anyway, below is a specular pass of the scene I've been using. And a word of caution. Clearly something looks a bit off with this specular pass. And I'm certainly not an expert in Carrara's render passes, since I tend to only use them for simple stuff, as other apps are far more flexible and easy for more complicated rendering and compositing jobs. And I doubt that many people access these render passes, so they probably don't get much scrutiny to the developers.
So maybe someone here would want to do some testing of some of the Carrara render passes to see how legit they are.
Now that we have a sense for the incredible flexibility and efficiency we can gain by using compositing, we need to understand the basics of what are called "Blending Modes".
We know we can isolate any aspect of our images....shadows, reflections, individual objects, diffuse and specular colors, UV information, and on and on.
So now that we have isolated those parts of our images, and made changes to those "layers", how do we composite them back together to obtain our final image (which, by the way, is also called the "beauty pass")?
Well, one thing we need to understand are blending modes. They define how the individual layers of our composition will be combined together to produce the final image. And they are, by the way, one more critical aspect of compositing that you probably won't learn by reading software manuals.
Blending modes basically define how each pixel in one layer will combine with each pixel in the next layer. And if I had the energy right now I'd outline the basics of blend modes, but I don't, so I won't.
Instead I'll give a challenge to anyone who wants to learn about compositing....
Since you can isolate every aspect of your image into its component parts (via render passes in your 3D app), it seems reasonable that you can then turn around and take those component parts and composite them together to obtain your final "beauty pass", which is basically the final render.
So how do you do that? How to you take the render passes, delete the final render image, and build that final render image solely from those component render passes?
And that's the challenge. Figure out how to take all the pertinent render passes that are provided by Carrara, and, using the appropriate Blend modes, build a complete "beauty pass" which contains all the components. It's a great way to learn the basics of compositing, and blending modes.
On your marks....get set....
Can you get a shadow pass to render correctly? I get a mess of random colored blocks that correspond to the render buckets. I know this was a known bug, and one of Sub7ths' big grievances with C7. wasn't sure if it was ever corrected in C8 or later.
What I get seems reasonable, but maybe not perfect. I'm scratching my head on some of it, but it's certainly usable.
Hi evil, yes it works fine. You can hit it with Gaussian blur to emulate soft shadows in places ;)
I have avoided it for years after trying it a couple times and getting the multi-colored tiles, however, I did find an old, low res test file with a shadow pass on my hard drive, and that looked fine. Maybe it was tied to resolution. I shall have to test it tomorrow. Gettin' a bit pooped right now.
Joe,
Thanks so much for this thread. I realize you do not have much time to conduct this tutorial but the time you are taking is extremely appreciated.
I find that sometimes I just don't know what a given filter is intended to do in a 2d application. Burn vs multiply vs screen, this is the sort of targeted insight that will help me out too. I will probably post an image or two and ask you to do some magic on it so I can learn what types of things people tend to think about.
Very useful thread. This is probably my favorite thread you've initiated, I can see myself gaining a lot from your generous offerings. Thanks and thanks again!!!!!!!
I echo Rashad's thoughts - this is seriously good work, presented in an easy to understand form for someone like me who has no formal grounding in 2D editors. Thanks, Joe:)
Something I'd like some clarity on is blending modes - does the blending mode in one layer refer how it blends with the layer below, or right through the stack?
Joe,
Have you ever run into a situation with compositing from Carrara where the 8-bits/channel limitation was not accomplishing a certain effect?
AE allows for 16-bits/channel and many rendering applications will save 16-bits/channel. OpenEXR seems to be a standard for this but without 16-bit/channel support, its not like it could be added to Carrara as another export format.
There is a 16-bit greyscale TIFF format that shows as an output choice from Carrara, however, only 8-bits of data are written (256 values) although the file is in 16-bit format, (which can contain 65536 values).
When I reported this many years back on version 7, it was reclassified as a feature request. I take this to mean there is currently NO 16-bit/channel export in Carrara. I have tested versions 7 up to 8.5 and the 16-bit greyscale TIFF option is still there, and still does not produce more than 256 values.
I believe someone (maybe Fenric?) said once, that under the hood, Carrara could work with 16 bit tiff files, but for some reason didn't export them. If I remember the discussion correctly, if you saved a scene with the Save Internally option selected, it would convert any image maps used in the scene to 16 bit tiffs. So even if you used .gifs for textures, they would be up-converted. So my question is, would it be that hard to actually allow Carrara to export that format?
Going back to your question about render passes: I don't know what Joe has experienced with the 8-bit passes, but I know that sometimes it can be very problematic with depth passes, particularly if an infinite plane is visible in the render. Strangely, if one is in the scene, but is obscured from view, then the pass will work (at least for me). I have also had depth passes fail that don't have an infinite plane and should seemingly work. When I say fail, I mean the pass produces a solid black silhouette.
Maybe it has more to do with the complexity of the image, and Carrara runs out of gray tones?
I guess one could create a rig with a near object and a far object and parent it to the camera to set up a range for the depth pass so it is consistent from frame to frame, if that is an issue. (I have not done it myself) Hopefully making the objects small enough so they did not actually render in the scene, or making them invisible (or transparent?) and still be calculated in the depth pass would work, but I don't know for sure.
Running out of grey tones is definitely an understatement. :) 256 grey levels is a far cry from 65536.
Thank you for this thread Joe, I certainly didn't know about the Object Index Render pass and having jumped into learning Fusion with just a very basic understanding of compositing, you're the second person I'll read who'll say to really learn about compositing first before learning the soft ware, so Brinkman's book is on my reading list, hoping I'm not ruined for learning already! Your challenge thrown here seems like it could be fun.......let's see......
This book by Brinkman "The science and art of compositing" looks promising. I downloaded it for free from the Internet:
http://www.sciencedirect.com/science/book/9780123706386
Now I'm looking forward to study it.
Here's what happens to my shadow pass: