Russ J.C. Image Analysis of Food Microstructure

Подождите немного. Документ загружается.

be used to reconstruct a view of just the particles, without the background or dirt

in the original image. This is done by convolution with the target image. (Convolution

is also carried out in Fourier space, and is exactly like cross-correlation except that

the phase angle of the complex values is not altered before multiplication.)

Correlation also offers a method by which the size and shape of the structural

unit that is repeated in the image can be determined automatically, without selecting

a target manually. In this case, the image is cross-correlated with itself (auto-

correlation). As shown in the example of Figure 3.43, the result provides a direct

measure of the size and shape of whatever structuring element is repeated throughout

the image. In the example, a thresholded contour line has been added to show the

outline of the representative feature.

To understand how this procedure works, it is again instructive to visualize an

equivalent process performed with the original image made up of pixels. A trans-

parent overlay with a copy of the image is placed on top of the original and is

gradually slid away from perfect alignment. How far can it be moved in each

direction before a feature in the overlay no longer covers itself in the original? The

cross-correlation mathematics calculates the degree of match and thus measures the

distance as a function of direction, to produce an averaged feature representation,

even though many of the features in the original image are partially covered because

of the three-dimensional nature of the surface. In the example shown, the size of

the structuring element in the protein isolate gel network was measured in this way

and shown to vary by about a factor of 2 as the CaCl

concentration was varied from

30 to 50 mM.

Cross-correlation has many other uses in image processing. One is to match

points in stereo pair images so that the lateral displacement (disparity) can be

determined and used to deﬁne the elevation of points. Depending on the type of

surface being imaged and the imaging method, this may be done either for every

point in each image or for just a few thousand interesting points (locations where

the variance is high and locally maximum), which are then used to interpolate a

smooth model for the surface. Cross-correlation is also used in tracking moving

features in a sequence of images, and for image alignment.

IMAGE COMBINATIONS

Several examples of the arithmetic combination of two images have already been

used in this chapter and the preceding one. For example, the removal of nonuniform

brightness by subtracting or dividing by a background image is a fairly common

procedure. In Figure 3.36 above, a binary image of the ﬁbers was combined with

the result of the Sobel direction ﬁlter to keep the brighter pixel. This replaced the

black ﬁbers with the direction value, while keeping the white background.

A full complement of arithmetic operations (addition, subtraction, absolute dif-

ference, division, multiplication, and keeping either the brighter or darker value) is

usually available, operating pixel by pixel to combine two images. Subtraction and

division, in addition to their use for background leveling, are often used to compare

images acquired through different colored ﬁlters or different imaging modes (e.g.,

in the AFM or SEM). Addition is primarily used as a way to compare the results of

2241_C03.fm Page 175 Thursday, April 28, 2005 10:28 AM

thresholding to the original image, as shown in Figure 3.38d above. It may also be

used to combine multiple wavelength images, for example a representation of the

visual monochrome brightness in a color image is often modeled as approximately

0.65 * Green + 0.25 * Red + 0.10 * Blue. Multiplication is infrequently employed

in image processing but is used in computer graphics to superimpose a texture onto

surfaces. Keeping the brighter or darker pixel values will be used extensively in the

next chapter to combine thresholded binary images with grey scale values. We will

also encounter there the various Boolean combinations (AND, OR, Ex-OR) that can

be used with two binary images.

All of these combinations require, of course, that the images be aligned with

each other. If they have originated from the same initial image, for instance by

different processing operations, then proper alignment is assured. If several original

images have been acquired, for example by recording different wavelengths, some

adjustment for misalignment may be required. In some cases, including medical

imaging with different modalities such as PET, CT, and MRI, or remote sensing

images recorded by different satellites, the image magniﬁcations, orientations, and

even points of view may be different.

For simple translation alignment of multiple images, the cross-correlation pro-

cedure shown above works automatically and quickly to provide the desired result.

If two similar images of the same subject are cross-correlated, the result is a peak

whose maximum is displaced from the center by exactly the amount of misalignment.

The center of the peak can be located by interpolation to a fraction of a pixel, and

the image shifted by that amount to produce proper alignment. Figure 3.44 shows

an example. This technique is also useful for aligning stereo pair images for viewing.

This type of shift alignment is also useful when dealing with video images of

moving subjects. The interlace scan used in standard video records each of the thirty

frames per second in two halves, or ﬁelds. These capture the even and odd scan

lines, respectively. If the scene is not stationary (either because the subject or the

camera is moving) this produces an offset as shown in Figure 3.45. Performing

cross-correlation between the two ﬁelds of the image allows correction of the

interlace offset. The assumption is that the entire scene is shifted uniformly; if

different objects within the ﬁeld have different motions the result is an average.

For situations in which shift, rotation, and perhaps perspective correction are all

needed to bring two images into proper registration, the basic procedure is the same

as for the correction of image distortion, shown in the preceding chapter. Generally

this requires the location of some ﬁducial marks in the images. In a few cases this

can be handled automatically. This may be possible in medical imaging by using a

few distinctive structures known to be present. A more common technique places

markers in the image, for instance by embedding wires or ﬁbers, or drilling holes

in a block of embedded material before cutting sections for microscopy. This tech-

nique is primarily used for alignment of serial section images for combination or

comparison.

The most common procedure is to rely on human knowledge and recognition,

and allow the user to mark registration points on the images. A minimum of one

point is needed for translation, two for translation and rotation, three for translation,

rotation, and scaling, and four to also include perspective correction. Some programs

2241_C03.fm Page 176 Thursday, April 28, 2005 10:28 AM

allow more points to be used either to determine the best ﬁt values of the needed

parameters, or to perform local image morphing to match one image to another.

Morphing is not usually used as part of an image analysis procedure, but it may be

useful in medical imaging to align the image of a subject to a standard template for

visual comparison.

(a)

(b)

FIGURE 3.44 Cross-correlation shift: (a) overlay of scans of two sides of a slice of swiss

cheese (one image ﬂipped side-to-side); (b) after cross-correlation alignment, subtracting one

from the other reveals the holes that do not extend all the way through the slice, which are

counted for use in the disector, a stereological routine that measures the number of holes per

unit volume.

2241_C03.fm Page 177 Thursday, April 28, 2005 10:28 AM

(a)

(b)

FIGURE 3.45 Correcting interlace shift: (a) original video image of bubbles in moving ﬂuid;

(b) after cross-correlation alignment of the even- and odd-numbered lines.

2241_C03.fm Page 178 Thursday, April 28, 2005 10:28 AM

In the illustration shown in Figure 3.46, the most common case of alignment

requiring translation, rotation and scaling is shown. Three points are marked on each

image, which deﬁne the required vectors. The result is a set of equations that give

the X, Y coordinates in the original image for each point X, Y in the transformed

(aligned) image. These are typically of the form

X = a + b X + c Y

Y = d + e X + f Y

(3.2)

and produce values for X and Y that are not integers. Hence it is necessary to

interpolate in the original image to determine the pixel brightness value to be

assigned to the new image. In fact, interpolation is commonly required for all of the

alignment or perspective correction procedures.

The simplest interpolation method is to use the nearest neighbor value, simply

truncating or rounding off the X, Y values to integers and using the corresponding

original pixel value. This preserves brightness data best, but causes aliasing of edges

as shown in Figure 3.47. Bilinear interpolation between the four nearest pixels, or

bicubic interpolation ﬁtting functions to the nearest 16 pixels, produce visually

superior results. More complex interpolation methods using sinc, spline, or other

functions are sometimes used in computer graphics but offer little advantage in

routine image processing applications.

Once images have been aligned, the various arithmetic operations can be applied

to combine them. In addition to these steps, the use of color channels (either RGB

or HSI) may be useful to reveal complex relationships. In the example shown in

Figure 3.48, both of these tools are used. The Fura stain reveals calcium activity.

Ratioing the image acquired with an excitation below the absorption edge of the

stain to that with an excitation above the edge measures the amount of calcium

activity. But the total amount of stain also has a distribution in the sample, which

can be measured with the sum of the two images. One way to show these results

together is to place the ratio image into the hue channel and the sum image into the

intensity channel of a color image (in the example, saturation is set to maximum).

As another example of image ratios and combining images for display, the ratio

of two ﬂuorescence images acquired at different times can be used to calculate

lifetimes. The ratio image is commonly displayed in the hue channel while the

intensity channel shows the original intensity distribution.

Tracking the motion of objects also uses image combinations. In the example

of Figure 3.49 a series of images taken over a period of time are combined by

keeping the darkest pixel at each location (because the features are darker than the

background). The combined image can be thresholded and skeletonized as described

in the next chapter to mark the path of each feature for measurement.

The most common purpose for image combinations is the detection of image

differences, usually by subtraction or ratioing. The differences may reﬂect changes

that have occurred over a period of time, or as imaged with different wavelengths.

Either serial physical or optical sections produce images from different depths in

the sample which can be compared for stereological calculations. Many of these

procedures are shown in this and other chapters in the context of the analysis.

2241_C03.fm Page 179 Thursday, April 28, 2005 10:28 AM

(a)

(b)

FIGURE 3.46 Three-point registration: (a) MRI; (b) PET; (c) CT scans of a human head.

The original three images are different in size and pixel resolution as well as orientation, and

three registration marks (arrows) were placed by hand based on visual judgment; (d) after

alignment, the images were combined as RGB color channels for viewing (see color insert

following page 150).

2241_C03.fm Page 180 Thursday, April 28, 2005 10:28 AM

(c)

(d)

FIGURE 3.46 (continued)

2241_C03.fm Page 181 Thursday, April 28, 2005 10:28 AM

(a)

(b)

FIGURE 3.47 Comparison of (a) nearest neighbor and (b) bicubic interpolation after image

scaling and rotation. Original image is a fragment of Color Figure 3.57; see color insert

following page 150.

2241_C03.fm Page 182 Thursday, April 28, 2005 10:28 AM

(a)

(b) (c)

(d) (e)

FIGURE 3.48 Fura stain to localize calcium activity: (a) excited at 340 nm; (b) excited at

385 nm; (c) sum; (d) ratio; (e) color composite as described in the text (see color insert

following page 150). (Courtesy of P. J. Sammak, University of Minnesota)

2241_C03.fm Page 183 Thursday, April 28, 2005 10:28 AM

THRESHOLDING

The image processing described in this chapter and the preceding one may be

performed for several different reasons. Improving the visual appearances of images

can assist the researcher in detecting and observing important details. Processing is

also used in preparing images for publication and to more effectively communicate

the important details to others who are less familiar with the subject. Processing can

be used to produce some measurement information directly, often by analysis of the

histogram. The other reason for processing is to prepare images for thresholding

(and subsequent quantitative measurement) of the features or structures present, by

producing an image in which brightness or color values are uniquely assigned to

the different structures. In the next chapter, processing and editing of the binary

images produced by thresholding is described, which is an indication that thresh-

olding is often an imperfect and not a ﬁnal step toward delineation of the structures

to be measured.

Thresholding in its simplest form is the process of selecting the pixels that make

up the structure of interest, on the basis that they have a unique brightness or color

range. These foreground pixels are then colored black and the remainder, the back-

ground, are colored white, producing a binary image. A word of caution: some

programs use the reverse convention in which the foreground is white and the

background black. Generally this indicates that they originated when computers

displayed white (or green, or amber, depending on the phosphor color) letters on a

dark cathode ray tube, and before the now-common white screens and black letters

came into widespread use. It makes no difference which convention is used (as long

as you know which is which). In any case, binary images are those in which there

are two kinds of pixels, which make up the features and the background.

(a) (b) (c)

(d) (e) (f)

FIGURE 3.49 Feature tracking by image combination: (a to e) individual images; (f) com-

posite with superimposed skeletons showing the paths of motion.

2241_C03.fm Page 184 Thursday, April 28, 2005 10:28 AM