Brinkmann R. The Art and Science of Digital Compositing

Подождите немного. Документ загружается.

252 The Art and Science of Digital Compositing

discuss this further in a moment. But if we simply want to view the images, it

may be easier to adjust the viewing device (the computer’s monitor) so that it is

darkened by a gamma of 0.45.

This adjustment will display the images at the

proper brightness without the need to preprocess every frame. As we mentioned

earlier, certain flipbook tools will make it even easier, allowing you to load a

specialized color table that modifies the way that images are displayed on the

system’s monitor.

For film images, a more complex nonlinear conversion is often used, which

takes into account various idiosyncrasies of how film stock responds to varying

exposure levels. The most common file format used to store film images is the

Cineon file format, which includes a specification for conversion into a nonlinear

‘‘logarithmic color space.’’ A logarithmic curve is again similar to the curve shown

in Figure 15.5. The Cineon file format is discussed in a bit more detail in Appendix

C, and an example image using Cineon encoding is shown in Plate 54.

Although you may have chosen to store images in a nonlinear format, you will

almost always want to convert these images back to linear space before working

with them because of the way in which color corrections will affect an image that

is stored in a nonlinear space. Consider the image shown in Plate 55, which is an

example of a simple color correction in which the red, green, and blue channels

have been multiplied by constant values of 1.2, 1.2, and 0.8, respectively. The left

side of the image was corrected in linear space, and comparing any pixel with

the original image will show a red value that has been reduced by 20%. The right

side of the image, however, was first encoded into a nonlinear color space (in this

example we used the Cineon specification), and then the same multiplication was

done. Once the color correction was applied, the image was restored to linear

space. As you can see, a fairly slight color correction in linear space has become

rather drastic when applied to a nonlinearly encoded image. In particular, notice

how the midtones have changed quite a bit more than the darker areas of the

image. This problem can be particularly vexing when working with bluescreen

elements—attempts to reduce blue spill may result in undesirable shifts in flesh

tones, for instance. Even the simplest image-combination tools can produce differ-

ent results when they are used on nonlinear images.

Although there are certainly operations that do not have this problem (geometric

transformations work equally well on images that are stored in any color space,

Although we state that you should ‘‘darken your monitor by a gamma of 0.45,’’ this does not

necessarily mean that you should explicitly set your monitor’s gamma at 0.45. Rather, you will need

to first determine what monitor settings are needed to give your monitor a linear response and then

add the adjustment on top of that setting. Different systems will have different (often poorly

documented) controls for adjusting the monitor to respond linearly to a video signal, and consequently

there is no way that we can tell you exactly what setting should be used.

Advanced Topics 253

for instance), in general the compositing artist would do well to heed the following

warning:

Color correcting or compositing images that are stored in a nonlin-

ear color space can easily produce unpredictable results. Always

linearize your images before manipulating or combining them.

WORKING WITH 3D ELEMENTS

In a number of places throughout this book we have mentioned issues that come

up only when working with synthetic, computer-generated 3D elements. There

are a few final items to be discussed, including some that relate to or can be

considered as part of the more advanced topics that are covered in this chapter.

The bulk of the general lighting and camera guidelines that we’ve discussed

apply equally well to the completely synthetic elements that are produced from

a 3D animation and rendering package. Usually the artist actually has more control

over a 3D environment than a real-world one. Synchronizing camera and lighting

setups will still be important in this environment, and even though different

software may have different conventions for how to specify these setups, there

is almost always a way to translate these settings into real-world units. The 3D

artist should thus be able to faithfully recreate the original background scene’s

environment and produce imagery that will integrate easily.

As we’ve mentioned elsewhere in this book, there are a number of features

that distinguish images that were created with a 3D system from those captured

via more traditional photography. The most obvious is, of course, the availability of

an explicit matte channel—a tremendous aid to the compositing process. Another

useful feature, the Z-buffer, will be discussed in detail in the next section. But

there are also a few additional idiosyncrasies with CG images that need to be

noted. First of all, remember that an image from a 3D rendering package has been

produced by a simulation of a camera, not the real thing. As such, real-world

characteristics such as lens distortions or film grain may not be included and

will need to be added during the compositing process. Remember too that the

brightness range of a CG image may be different from a live-action image. Very

often a scanned film image will never have any digital values below a certain

threshold, yet a digital image may actually have values down to zero. Integrating

the two will require that the contrast range in the CG image be adjusted, boosting

the blacks so that they match with the live-action plate.

When working with 3D elements, there will always be situations in which one

can choose to either modify the rendered element in the composite or to modify

the color and lighting parameters in the 3D scene and rerender the element from

scratch. Deciding which route to take may be dictated by practical considerations

254 The Art and Science of Digital Compositing

such as the amount of time available, but even assuming that these things are

equal, the decision should still be on a case-by-case basis. Within the 3D environ-

ment there is a much greater amount of control over how an object is lit than

there is once it has been rendered into an image. Unlike in 2D, where changing

the apparent direction of a light source can entail a great deal of effort, in the 3D

world one can easily move and modify lights at will. Thus, any significant changes

in lighting direction will usually require a rerender. Changes in the overall color

balance of an element, on the other hand, may be something that is done just as

easily in the composite. Rarely is the decision a trivial one, and often you will

find that the best method is to attempt the changes first in 2D and then revert to

the 3D solution as needed. Some of the more sophisticated software packages on

the market will actually offer the user a hybrid 2D/3D solution, wherein certain

parameters that would normally only be available in a 3D package are stored in

a specialized image file. Using such a system, the compositing artist can directly

manipulate such things as the relationships among the ambient, diffuse, and

specular illumination on an object. This integration between 2D and 3D will

continue to become more common as time goes by.

Working with 3D elements may also present some additional logistical prob-

lems, particularly if your 3D elements are being created by someone else. When

dealing only with a collection of live-action plates, you know that the elements

you are given will remain the same for as long as you are working with them.

When integrating 3D, however, the situation may not be quite so stable. It is very

common for the development of a particular 3D element to be something that is

happening concurrently with the integration of that element into a scene, a process

that has both benefits and drawbacks. The benefits are that it will give both the

compositor and the 3D artist the ability to address problems by rerendering as

necessary. There may even be times when a compositing script can be made more

efficient by removing certain operations (such as a color correction) and applying

a compensating operation to the 3D element itself as it is being rerendered. Of

course the significant downside to this process is that the compositing artist may

spend a great deal of time balancing a particular element into a scene and then

suddenly be confronted with a radically changed new element provided by the

3D artist. The only way to avoid this is to ensure that the communication between

the 2D and the 3D artists is thorough and timely.

With the growing popularity of completely computer-generated movies, televi-

sion shows, and theme-park rides, there is now a good deal of compositing work

being done without any live-action elements involved. Although it is possible for

an entire CG scene to be rendered as a single finished shot, experienced 3D artists

will still tend to render more complex shots as a series of discrete layers and then

composite these layers together into a final scene. This methodology allows the

3D artist to avoid time-consuming rerenders of large scenes when there is only

Advanced Topics 255

a problem with a specific element. Instead, only the element in question will need

to be redone, usually saving a great deal of time and resources.

Z-Depth Compositing

There is yet another useful technique that can sometimes be used when dealing

with synthetic CG images, a method that gives the compositing software additional

information about the spatial relationships among objects in the 3D scene. This

technique is known as Z-depth compositing.

Whenever a 3D database is created for the purpose of rendering a CG image,

every object in the scene will have a specific color, material, and illumination

assigned to it. When it comes time to render a scene from this database, each

pixel in the image that is generated will correspond to a certain point on one of

the objects in the scene. But there is more information in this database than just

the color and lighting description of the scene. The spatial relationships of the

objects in this scene are also very well defined. Every object in this virtual scene

has a specific location in virtual space, and the 3D software is obviously able to

determine these locations with great accuracy. Z-depth compositing (or sometimes

just ‘‘Z-compositing’’) uses a special image that is explicitly created to quantify

these spatial relationships, thereby incorporating depth information into the com-

positing process. In addition to the standard color image that is rendered for a

scene, software with Z-depth functionality allows you to turn on a special mode

that can render a second type of image. As with a matte image, this new type of

image requires only a single channel to represent its data. But instead of using

pixels to represent transparency information for the corresponding point in the

color image, each pixel specifies the spatial location for each point.

The image that is generated is known as a Z-depth image, based on the typical

axes that are used to measure space in a 3D coordinate system (X is horizontal,

Y is vertical, and Z is depth). Depending on the tools you are using, this Z-depth

image may be kept as a separate file, or it may be integrated with the color image

as an additional channel. When integrated, it is referred to as the Z-channel or

depth channel of the image. Consider the image shown in Figure 15.6a, a simple

synthetic scene. Figure 15.6b is the corresponding Z-depth image for this scene.

We can choose to represent spatial information in a few different ways, but in

general we are interested in the specific distance any given point is from some

fixed reference point. This reference point may be an absolute location in our 3D

space—usually the origin—or it may be relative to a given object, usually the

camera. For the sake of this discussion we’ll consider the case in which we are

only interested in the distance from the camera. Areas in the image that are closer

to the camera are represented with darker pixels; those that are farther away are

brighter.

256 The Art and Science of Digital Compositing

(a) (b)

Figure 15.6 (a) A simple 3D rendered scene. (b) The Z-depth image for this scene.

This depth image can be useful in a variety of ways. At its most simple, it can

be combined with the original image in order to add some depth-based at-

mospheric effects that increase with distance. But most software will allow us to

use much more powerful tools in conjunction with the Z-depth image. Since we

now have accurate information about the depth of the various objects in the scene,

the process of determining occlusion can be automated. In a simple scene this

ability may not save us a great deal of time, since we could have already generated

a matte for the various objects and have manually determined the foreground/

background relationship between anything new that we want to add to the scene.

But if our scene becomes more complex, automated Z-depth compositing can be

incredibly useful. Consider a second image such as Figure 15.7a, which is also an

image with a large number of objects in it at a variety of distances from the

camera. If we have Z-depth information for this image as well (Figure 15.7b),

the software can automatically determine the proper foreground/background

relationships for each of the different objects in the scene. Figure 15.7c is the result

of our Z-composite of Figures 15.6a and 15.7a. This same step, had it been done

manually, would have required a great deal of time to individually determine

which objects should overlap.

It is important to recognize that there are several limitations to Z-depth compos-

iting. The most obvious is that this Z-depth information is generally unavailable

for any live-action elements (more on this in a moment), and thus Z-compositing

is typically useful only when combining CG elements. But even within a wholly

synthetic scene there are limits to the accuracy of this Z-depth information. Earlier

we mentioned that each pixel in the Z-image will correspond to a point on a

particular object. But this is not always the case. If we have a partially transparent

object in the scene, the pixels for this object in our color image will receive color

Advanced Topics 257

(a) (b)

(c)

Figure 15.7 Z-depth compositing. (a) A second 3D rendered scene. (b) The Z-depth image for

this scene. (c) The Z-composite of Figures 15.6a and 15.7a.

information from both the foreground (semitransparent) object as well as from

any object that is behind this object. How do we assign a depth value for such a

pixel? The short answer is, we can’t, at least not usefully. We could choose to use

the depth values of the foreground object (which is usually what is done), but if

we tried to use this depth value to add a new, more distant object into the scene,

our Z-compositing software would automatically put this new element behind

the first object. But the first object, being partially transparent, is already showing

a bit of the original background behind it. Thus, the foreground object would

258 The Art and Science of Digital Compositing

appear to be in front of the new element, but would still be revealing a bit of the

original background.

An example of this problem is shown in Figures 15.8 and 15.9. Figure 15.8a

shows our scene, and Figure 15.8b shows the depth information for this scene.

As you can see, the semitransparent sphere is fairly close to the camera, and the

checkered wall is farther away. Now, let’s attempt to compose the object shown

in Figure 15.9a into this scene. The depth information for this object is shown in

15.9b. Based purely on the depth information, a Z-composite would produce the

final image shown in Figure 15.9c. As you can see, the result is unacceptable. The

final image should look like Figure 15.9d, where the new object is still visible

through the foreground sphere. Unfortunately, there is really no way to fix this

problem, short of rerendering the scene as an integrated whole or rendering all

(a) (b)

Figure 15.8 (a) Scene containing a semitransparent object. (b) The Z-depth image for this scene.

(a) (b)

Figure 15.9 (a) The new element. (b) The new element’s Z-depth image.

Advanced Topics 259

(c)

(d)

Figure 15.9 (c) An unsuccessful attempt to composite 15.9a into Figure 15.8a based purely on

depth information. (d) Proper integration of the new element.

the elements individually. Consequently, it is generally safe to say that Z-depth

compositing will be problematic and in some cases even useless if there are

transparent objects in the scene with which you are working.

Unfortunately, even if there is no explicit transparency involved, the edges of

certain objects can sometimes cause a problem because most 3D objects are actually

260 The Art and Science of Digital Compositing

rendered with antialiased edges, which effectively introduces a small amount of

transparency in these areas. A pixel that is precisely on the edge of an object will

get part of its color from the foreground object and part from the background.

This will always present a problem unless your object was rendered in front of

a neutral (or black) background and you have an additional matte channel to define

the transparency of these edges. Even if your foreground object is overlapping

something at a different depth, the background information that is mixed with

the foreground edge may not be terribly noticeable, particularly if your foreground

object is in very sharp focus. Often a bit of additional edge processing can help

to alleviate any artifacts that are introduced. In general, though, it is safe to say

that whenever there are problems with Z-depth compositing, they usually involve

either transparency or object edges.

Another inherent problem with the Z-compositing process is the fact that there

are no standards for exactly what depth or distance corresponds to a particular

brightness value in the Z-image. In fact there really cannot be any standard, since

two different scenes may have wildly different distances from the camera. Usually

when Z-depth images are rendered, one specifies a particular depth range that

will be represented, and all pixel values in this image will be normalized for this

range. But this range might be the equivalent of only a few meters of depth in

one image, whereas another image may have depth values that would be measured

in thousands of miles. It is not uncommon to spend some time normalizing two

different Z-depth images to each other before they can be combined.

Precision and bit depth is also an issue. If your system is only using 8 bits to

represent the Z-depth values, then you will only have 256 different depths that

can be identified. Sixteen-bit Z-depth images will be better, but the best systems

use floating-point values, which provide for much higher precision as well as far

more flexibility for the range of depth values that are used.

As we stated earlier, Z-depth compositing is primarily used with computer-

generated images, since this is the only type of image for which accurate depth

values can be automatically generated. But it is possible to occasionally make use

of these techniques even when live action is involved. Most commonly this is

done by arbitrarily assigning a constant depth value to a particular live-action

object and then integrating it with a synthetic element. If we create a Z-depth

image for our live-action element, we can choose an overall depth for that object.

Assigning a medium-gray value for the Z-image of our live-action element would

cause the Z-depth compositing software to automatically place this live-action

element about half-way back in our 3D scene. In certain situations, it may even

be worthwhile to hand-paint a more detailed Z-depth image for a particular object.

This will, of course, require someone who is skilled enough to determine the

relative depths of the different objects in the scene, but the time savings may be

worth the effort.

Advanced Topics 261

Research is being done on allowing Z-depth information to be automatically

extracted from live-action footage. Stereoscopic photography inherently contains

some depth cues, and even a single camera view can give some information if it

is moving enough to introduce parallax shifts. Sophisticated software algorithms

are able to analyze such imagery and can often produce useful depth information

for the scene. As the availability and reliability of these tools increases, expect to

see more and more use of Z-depth compositing in conjunction with live-action

footage.

RELATED 2D DISCIPLINES

In the very first chapter of this book, we alluded to the fact that there is 2D work

that does not fall into the category of digital compositing. We will round out this

chapter with a look at a few disciplines that share many things in common with

pure digital compositing but should usually be considered as separate fields. The

following processes share some conceptual similarities with digital compositing,

and you will find that they are often used in conjunction with digital compositing

as part of the overall process of producing a finished set of imagery.

Morphing

In Chapter 3 we briefly looked at tools that can be used to warp, or distort, an

image. Elsewhere, in Chapter 4, we covered some image-combination tools that

could be used to dissolve between two different images over a period of time.

Although both of those processes are used quite often by themselves, there is also

a combination process that uses both techniques to implement a very specialized

transition between two different images. This process is known as morphing.

Morphing, in theory, is really quite simple, although the process of creating a

good morph can be quite time-consuming. It is essentially a two-step process,

although the steps will not necessarily occur sequentially. Morphing is considered

a specialized transition not only because of the complexity of the tools that are

used, but also because it really only works effectively if the two elements that are

involved in the transition have some reasonably similar characteristics in common.

It was developed, and is still primarily used, as a method to make it appear as

if one object were physically transforming into another.

The first step in the process is to identify which key features in the first element

will correspond to features in the second. Consider the example images shown

in Figure 15.10. Let’s attempt the classic morph scenario, in which we wish to

make the skull appear to transform into the smiley face over a series of frames.

For the first step of the morph, warping tools are used to generate two new

sequences of images. The first sequence consists of the skull being warped over