Russ J.C. Image Analysis of Food Microstructure

Подождите немного. Документ загружается.

IMAGE SHARPENING

The visual appearance of image detail is improved by increasing the amount of

contrast at well deﬁned edges and steps. This result is often accomplished by

applying high pass ﬁlters to the data, meaning that high frequencies are kept or

increased in amplitude while low frequencies are reduced or eliminated. The termi-

nology corresponds to performing the processing in Fourier space, but in practice

the operations are often performed directly on the pixel array by the use of convo-

lution kernels — arrays of weights that are multiplied by the pixel values and the

results summed to produce a new value for the pixel. Kernels were introduced in

the preceding chapter as a way to perform smoothing, which is a low pass ﬁlter that

reduces high frequencies — in that case random speckle noise.

To better understand how these ﬁlters work it is instructive to start with a simple

derivative. Unlike the smoothing ﬁlters shown previously, the weight values in these

kernels are neither symmetric about the center nor all positive. A representative ﬁrst

derivative kernel is an array of the form

The operation of this kernel is to replace the value of each pixel with the

difference between the neighbors above and below, hence forming a vertical deriv-

ative. The weights perform some averaging in the horizontal direction to reduce

random noise. Because of the negative weights, it is possible for the result of applying

this kernel to be negative. Since the display can usually only show values in the

range 0 (black) to 255 (white), the result is usually divided by the largest number

in the array (2 or 4 in the examples) and has a medium grey value of 128 added to

it so that negative or positive results are visible. (A few programs perform the

additional step of autoscaling the result to maximize contrast without clipping.)

Figure 3.6 shows an example. The effect of the derivative is to produce an embossed

appearance in the image, in which horizontal edges have a bright and a dark edge.

Vertical detail is not enhanced, and in fact virtually disappears.

A directional derivative may be useful for eliminating the visual distraction

produced by strong shadows, scratches, or other directional information superim-

posed on an image. In the example shown in Figure 3.7, the grains in the surface

of aluminum foil (used as food packaging) are obscured by the linear marks of the

extrusion or rolling process. Applying a directional derivative exactly parallel to the

marks makes them disappear so that the underlying structure is easier to examine.

While these embossed images may be visually attractive, they are only rarely

useful for measurement purposes because of the directionality of the enhancement.

A second derivative can be similarly constructed which is non-directional. A kernel

of the form

+1 +2 +1 +1 +2 +4 +2 +1

000 or 00000

–1 –2 –1 –1 –2 –4 –2 –1

0–1–2–1 0

–1 –1 –1 –1 –1 +3 –1 –1

–1 +8 –1 or –2 +3 +8 +3 –2

–1 –1 –1 –1 –1 +3 –1 –1

0–1–2–1 0

2241_C03.fm Page 135 Thursday, April 28, 2005 10:28 AM

(a)

(b)

FIGURE 3.6

Image of a beef roast (a) and the result of applying a vertical derivative (b).

2241_C03.fm Page 136 Thursday, April 28, 2005 10:28 AM

is a classic Laplacian operator. It is necessary to add an offset value to allow negative

results to be shown for the derivative. By taking the difference between a central

pixel (or a small central region) and its surroundings, the Laplacian is sensitive to

brightness changes that occur in any direction. The result, as shown in Figure 3.8,

is to keep only the changes in brightness and to mark each one with a black and a

white edge.

The problem is the same as for local equalization in that the overall contrast is

lost, but this can be restored by adding the result to the original image. The usual

short cut for that operation is to add 1 to the weight for the central pixel (and

eliminate the offset value). That produces the most widely used sharpening ﬁlters,

which increase the amplitude of brightness changes without altering the overall

contrast as shown schematically in Figure 3.9. This sharpening operation is com-

monly applied to photographs before printing them, because most printing operations

cause a certain amount of blurring, which the ﬁltering corrects.

These sharpening ﬁlters do not actually sharpen the image in the sense of

correcting out-of-focus blur, as described in the preceding chapter. What they do is

trick the eye by increasing contrast where it already exists. This is almost entirely

used for improving the visual appearance of images, and not for facilitating any

subsequent measurement or analysis of the images.

The sharpening ﬁlter shown above has two important limitations: it is sensitive

to speckle noise (a pixel that is slightly different from its neighbors has that difference

increased) and it only increases the contrast for pixels that differ from their imme-

diate neighbors. A more general technique for sharpening is the so-called “unsharp

mask” ﬁlter, which is itself a specialized version of the most general form of this

approach, the difference-of-Gaussians (DoG) ﬁlter. This latter procedure is believed

to be similar to the processing performed by neural networks in the human retina,

and can also be shown to correspond to a band-pass ﬁlter performed using Fourier

transforms.

(a) (b)

FIGURE 3.7

Image of the surface of an aluminum foil (a) and the result of application of

an embossing (derivative) ﬁlter parallel to the extrusion or rolling lines to hide them (b).

2241_C03.fm Page 137 Thursday, April 28, 2005 10:28 AM

(a)

(b)

FIGURE 3.8

The beef image from Figure 3.6 after application of (a) Laplacian and (b)

sharpening ﬁlters.

2241_C03.fm Page 138 Thursday, April 28, 2005 10:28 AM

The DoG method (Figure 3.10) uses two copies of the original image. One is

blurred with a Gaussian ﬁlter having a very small standard deviation, typically

ranging from less than 1 to a few pixels, in order to reduce random pixel variations

(noise). The second is also blurred with a Gaussian ﬁlter with a standard deviation

about 3 to 5 times larger. The larger blur is intended to selectively remove important

edges and detail from the image. The difference between the two images isolates

those important edges and detail. As for the other sharpening and enhancement

methods, these details are then often added back to the original image for viewing.

This general approach of ﬁnding a way to remove the important information from

an image and then recovering it by subtraction from the original is a useful technique

in many situations.

In the particular case in which the original image does not contain much random

noise, the ﬁrst (small) blur is unnecessary and the DoG method reduces to the unsharp

mask. In this case a blurred copy of the image is subtracted from the original. The

name originates in the photographic darkroom, describing a century-old practice.

FIGURE 3.9

Diagram showing the increase in contrast at edges produced by sharpening

operations.

(a) (b)

(c)

FIGURE 3.10

Light micrograph of stained tissue (the original image is in color, but the process

is applied only to the intensity or luminance channel): (a) original (b) Laplacian; (c) DoG.

Original

intensity

profile

"Sharpened"

Intensity

Profile

2241_C03.fm Page 139 Thursday, April 28, 2005 10:28 AM

Photographic ﬁlm has a much greater latitude than paper prints, and a negative can

therefore capture bright and dark portions of a scene that are difﬁcult to show in the

ﬁnal print. One solution is to ﬁrst use the negative to expose a second piece of ﬁlm, at

a 1:1 scale but slightly out of focus. Developed, this is the unsharp (because it is out

of focus) mask. Placing the two pieces of ﬁlm together and printing through both

produces reduced overall contrast, because the mask is dark where the original negative

is light and vice versa, but preserves edge detail because the mask is not sharp. The

computer procedure corresponds to that of the darkroom, but with the added conve-

nience of interactively controlling the amount of blur and the amount of subtraction.

The effect of a DoG or unsharp mask ﬁlter is to increase the magnitude of contrast

at edges and other ﬁne detail, as indicated in Figure 3.9. Some programs allow selec-

tively keeping the added contrast on either the bright or dark side of the edge, which

can also further improve the visual appearance of the result (Figure 3.11).

The DoG and other sharpening ﬁlters are generally described as “hi-pass” ﬁlters

because in terms of the Fourier transform and the frequency-space representation of

the image, they reduce the amplitude of low frequency terms that control global

contrast while keeping the higher frequency terms that produce local detail. The

DoG reﬁnes this approach further by reducing the very highest frequency terms

(a) (b)

FIGURE 3.11

Unsharp mask or DoG applied to an image of a cut surface through cheese

(image courtesy of A. J. Pastorino, Department of Nutrition and Food Sciences, Utah State

University): (a) original; (b) adding only the dark edge contrast; (c) only the light edge

contrast; (d) both.

2241_C03.fm Page 140 Thursday, April 28, 2005 10:28 AM

(corresponding to the random pixel noise). That suggests that the greatest control

over detail enhancement can be effected by working directly with the Fourier transform.

It is possible to construct a ﬁlter that keeps whatever range of frequencies corresponds

to the most interesting image detail while suppressing others that represent noise or

unwanted global contrast, much in the manner of adjusting an audio equalizer to

get the best sound from a given set of speakers. Figure 3.12 shows that this method

can enhance detail and at the same time suppress unwanted contrast so that infor-

mation can be recovered from deeply shadowed pits in an SEM image.

This type of range compression is particularly useful for images in which

different regions are very bright or very dark. As noted previously, human vision

detects local changes in brightness of a few percent, and largely ignores large scale

differences in brightness. A method that compresses the large scale variations while

increasing the local differences makes details in bright or shadow areas more visible.

SEM images of rough surfaces, such as the example in Figure 3.5, can often be

improved by this technique.

(a) (b)

(c)

FIGURE 3.12

SEM image of processed Colby cheese (a) the original image appears in

Aguilera and Stanley, Figure 4-2f, courtesy of the authors and Ken Baker, Ken Baker Asso-

ciates and (b) the result of applying a bandpass ﬁlter. (c) Each slider in the equalizer adjusts

the amplitude of frequencies higher by a factor of 2.

2241_C03.fm Page 141 Thursday, April 28, 2005 10:28 AM

FALSE COLOR AND SURFACE RENDERING

The reason that sharpening methods are useful to visually enhance detail in

images lies in the limitations of human vision, which does not notice gradual changes

in brightness nor variations of less than a few percent. Another very popular way

to make small differences in brightness visible is to assign false colors to the image.

For an 8 bit image with 256 stored brightness levels, it is straightforward to construct

a table of 256 colors (usually called a “CLUT” or “Color Look Up Table”) and

assign one to each of the possible brightness levels. Human vision can discern only

about 20 to 30 brightness levels in an image, but can distinguish hundreds of colors,

so every different pixel value can be readily observed.

Because it emphasizes the difference between pixels, the application of false

color (pseudocolor) does not help the observer to see the relationship between

features in an image. False color images are very good for discerning gradual changes

in brightness in large, nearly uniform areas. Applied to a typical image, they may

act much like camouﬂage and actually hide structure by breaking it up. Applying a

CLUT in which the colors vary gradually, for instance in a spectrum or heat scale,

causes less break up in the Gestalt of an image but also does not provide as much

ability to visually distinguish small changes. These pictures do, however, remain

popular with magazine editors who want a splash of color for the front cover.

While human vision is a poor judge of changes in brightness or color, we do

have millions of years of evolution to guide the interpretation of surface images.

The appearance of light interacting with a surface, particularly if the angle of

incidence of the illumination and the angle of viewing can be altered, is instinctively

interpreted to provide understanding of the surface geometry and roughness, color,

and reﬂectivity. The mathematics of light scattering and reﬂecting from surfaces is

well understood and computer programs are routinely used in CAD applications,

movie animations, and advertising to model the appearances of physical surfaces.

Using the same math to generate a rendering of an image as though it were a surface

often reveals details that were difﬁcult or impossible to see in the original image.

There are two parts to surface displays. One is to create a geometric represen-

tation of a three-dimensional surface (or something we are pretending is a surface)

on a two-dimensional computer screen. Geometrical modeling produces a perspec-

tive-corrected view from a chosen point in space in which locations are displaced

upwards according to some value (usually pixel brightness) to represent the surface.

Surface rendering then controls the brightness of that point based on the local angle

of the surface facets, the intensity and position of the incident light, and the surface

characteristics, speciﬁcally how much of the light is specularly reﬂected from the

surface and how much of it is diffusely scattered. Figure 3.13 illustrates these

possibilities for enhancing the visibility and interpretability of detail.

Many atomic force microscopes offer at least geometric modeling, and some

include surface rendering of the image, to show their images. In many cases, the

geometry revealed in the images is literally the elevation as a function of position

(Figure 3.14), which is one of the major measurements that the microscope provides.

Since AFM instruments can also record many other characteristics of the sample-

tip interaction (e.g., lateral force, tapping compliance, heat conductivity, etc.), each

2241_C03.fm Page 142 Thursday, April 28, 2005 10:28 AM

(a) (b)

(e) (f)

FIGURE 3.13

Scanned probe image of the surface of skin: (a) original (image width 1 cm);

(b) the surface Phong rendered; (c) a geometric model constructed from the data; (d) the

geometric model with superimposed surface contrast rendered with diffuse lighting; (e) same

as (d) but with specular lighting; (f) the surface color coded to indicate elevation, with

superimposed contour lines (see color insert following page 150).

2241_C03.fm Page 143 Thursday, April 28, 2005 10:28 AM

of these can be shown as though the values represented a surface, as well. Adding

color and rendering of the surface (usually making the simplifying assumption that

the surface is a highly specular reﬂector) can also be used to help reveal subtle detail

to the eye.

As shown in Figure 3.15, presenting grey scale intensity as though it represents

a surface is also useful with other types of images. The example shown is a ﬂuo-

rescence image of stained tubules. When the intensity is interpreted as surface

elevation and the resulting data rendered, the faint markings may become easier to

see and the differences in intensity easier to interpret.

FIGURE 3.14

AFM image of the surface of chocolate with a rendering showing the physical

geometry of the actual surface. (Courtesy of A. J. Pastorino, Department of Nutrition and

Food Sciences, Utah State University)

(a) (b)

FIGURE 3.15

Rendering the ﬂuorescent intensity of tubules (a) as a surface (b), to better

see faint marks and compare intensities.

2241_C03.fm Page 144 Thursday, April 28, 2005 10:28 AM