Caballero B. (ed.) Encyclopaedia of Food Science, Food Technology and Nutrition. Ten-Volume Set

Подождите немного. Документ загружается.

to provide information about how the sensory char-

acteristics of products, ingredients, or other related

information and services are related to perceived

quality characteristics and consumer liking. How do

you make a peanut butter ‘creamier’ or a facial tissue

‘softer’? How do you describe the fragrance of a fine

perfume? How easy is it to push the pump on a spray

deodorant? When does a potato chip start to taste

‘stale’? These are just some examples of what sensory

analysis can do.

Setting Objectives

0002 Given numerous applications of sensory analysis, the

practical considerations associated with effective

testing must begin with the objectives. In short,

what information is needed, or ‘What do you want

to know?’

0003 Consider the following hypothetical situation. You

own an icecream company. Your supplier of vanilla

tells you to expect a substantial increase in price. You

do not want to make consumers pay more for

your icecream, so you find a different provider at

a lower price. Your biggest concern is providing

the same great flavor to your consumers. Therefore,

an initial question is: ‘If the new vanilla is used,

can people distinguish between the products?’ or ‘If

I use the new vanilla flavor, do I still have a great

product?’

0004 A key practical consideration is to realize that those

two questions are fundamentally different. The first

question is an issue of difference; the second is an

issue of liking or preference. If you decide that people

should not be able to detect any difference in the

product, you have set a very different criteria than if

you determine that the product can taste different as

long as consumers like the new product as much as, or

more than, current product.

0005 Part of setting the objectives in a sensory test is risk

management. Every person who will buy the product

will not be asked to test the product (see section

Resource Management); nor will it be tested in

every situation in which people eat icecream. You

have to identify an objective and select a test protocol

that addresses how much risk you are willing to

accept. If you want to be especially sure that a con-

sumer will not find any differences in the icecream,

you need to specify that in the objective and use a

large population of people who are able to discrimin-

ate differences in icecream. If you just want to be

pretty sure that you still have a good icecream that

will be acceptable to the average user, specify this

objective and test a population of typical users

to make sure that the icecream still scores high in

acceptance.

0006Deciding on the objective(s) of the sensory test and

getting everyone to agree on the objective(s) is the

first practical consideration in sensory testing. If this

is done poorly, the rest of the test may be meaningless.

If the objectives are set correctly, and everyone has

agreed in advance, the test and the reporting of results

will be much easier.

Resource Management

0007The next practical consideration in sensory testing is

managing the test to provide the key information in a

cost-effective, timely way. Although it is possible to

conduct sensory studies in many ways, the best ways

will be those that provide appropriate answers to key

questions as quickly as possible using the minimum

amount of resources. It is very easy to overplan or

overconduct studies. Commonly, tests are conducted

that either are not required or that try to answer too

many questions simultaneously.

0008If we go back to the icecream company, an initial

question might be, ‘Is there an obvious difference if we

change to the new vanilla?’ If there is, then difference

testing is a waste of time. Initial benchtop screening

can help to answer those questions. Although lab

personnel and other employees are not representative

of the typical user population, they can be a cost-

effective way to initially screen products to determine

whether further testing is needed.

0009If the company decides, after benchtop screening,

that testing is needed, an appropriate test is required.

While it might be nice and interesting to find every

consumer of our icecream and ask them to taste the

two samples, such an enormous study would make

little sense from the perspective of cost or time and

would provide no information that we cannot gain

from a smaller more focused study. It would be a huge

waste of resources to ignore the practical issues of

resource management.

Sampling

0010Another key practical consideration is sampling of

the population. It is important that participants in

sensory test are the people we really need to answer

the questions. Caution must be exercised that the

testers do not have biases that would affect their

testing. Thus, the first step in obtaining a group of

people who are representative of the product’s con-

sumers is acknowledging that employees of the com-

pany often do not meet these criteria. The potential

biases, both good and bad, that employees may bring

about their companies products often disqualify them

from making a real assessment of the quality of the

product. Although employees provide a cheap and

SENSORY EVALUATION/Practical Considerations 5137

convenient source of people to test, and may be suit-

able for a screening test, their views should not

be used as a final decision-maker when deciding

what consumers really think about the product(s) in

question.

0011 For descriptive sensory tests, the appropriate people

are those who have a good ability to sense attributes

and have been trained in some way to describe what

they are testing. Panelists trained to have a common

vocabulary make the task of translating sensory per-

ceptions into meaningful information for product

scientists much easier.

0012 Consumers usually are screened to help assess their

potential to represent either potential users or current

users of the product. Screening questions may include:

age, gender, product related behaviors (e.g., use in the

past 3 months), or attitudinal and other factors that

help to identify people who are likely to use, or do

use, the product or other products in the category.

0013 There is no ‘magic’ minimum number of consu-

mers required for any specific consumer test. The

number of respondents should be large enough to be

representative of the consumer population and to

provide the statistical power needed to differentiate

the products. This is a fine balance. The tendency is to

test too few people ‘to control costs’ or to test too

many people ‘just to be sure.’ Historical information

on the consumer population, often obtained from

marketing or purchase data, and information on stat-

istical variability can help to determine the number of

consumers for the test that balances resources and

risk management.

0014 The second major part of sampling is the product.

The range of products or samples tested can have an

impact on the test. For example, testing a high-fat

vanilla icecream with a reduced calorie vanilla ice-

cream will result in different sensory scores than

testing the high-fat product against other similar

high-fat products. In the end, the results obtained

from testing are dependent on the samples evaluated.

0015 The second major concern in product sampling is

making sure that the product being tested represents

the product that actually is manufactured. Too often,

incorrect conclusions are drawn from a test because

‘early prototypes’ (that is, samples that are different

from the actual final product) are used in the test, for

example, testing a product from one manufacturing

plant when three plants product the product using

different equipment and ingredients suppliers, or

when products have been mishandled. In those and

similar cases, the products in the test are not repre-

sentative of the product that consumers actually see

when they ‘purchase off the shelf.’ If the product does

not really represent ‘the product,’ the test results are

meaningless.

0016Issues associated with sampling are critical prac-

tical considerations in sensory testing. Ensuring that

the right product is tested and that the correct popu-

lation of people tests the product are essential to

providing actionable data.

Analysis and Reporting

0017Time and resources are limited, finite properties.

Within this context, the use of appropriate analysis

and reporting is the essence of practical consider-

ations for sensory analysis. Proper analysis and ap-

propriate reporting of well-conducted studies provide

timely information that is actionable and concise.

0018The use of appropriate data analysis is an extension

of the first practical consideration – clear objectives.

‘How is this analysis related to the specific object-

ives?’ is a question that should precede all statistical

applications. One advantage of basing analysis and

reporting on specific objectives is that it refines the

scope of analysis that is appropriate. For example,

analysis of results from common difference tests

may require only the use of standard tables, whereas

analysis of results for data that attempt to ‘explain’

liking data on the basis of descriptive sensory proper-

ties is more complex. Appropriate analysis concludes

with obtaining all findings that are directly relevant

to the stated objectives.

0019At the end of almost every study, people want three

basic questions answered: ‘What did we learn?’, ‘Why

is this information important?’, and ‘How can we use

this information?’ In reality, after testing is complete,

and the data have been analyzed, some findings are

not as important as originally thought, some will

require further investigation, and some may provide

several possible courses of action. Within this con-

text, quality reporting, whether written or oral, pro-

vides specific, concise answers with supporting details

when needed. Reporting that leads the audience

to wonder ‘so what?’ is misguided and will be inef-

fective. Further, quality reporting does not mean that

everyone is happy with the answers: it means

that everyone is clear on what the answers are, the

relative confidence in these answers, and how they

are actionable.

0020The basic purpose of a report is to communicate

information related to specific goals. Thus, clarity

and brevity are key considerations in appropriate

reporting. If the relevance of the information pre-

sented cannot be clearly understood by the reader or

listener, the value of the work will not be maximized.

Further, acknowledgement that people have a finite

attention span is key to effective communication,

especially for complex sensory information. This em-

phasizes the importance of stating key findings early

5138 SENSORY EVALUATION/Practical Considerations

so that the audience can focus on the meaning of the

information presented.

0021 Sensory analysis can answer a wide range of differ-

ent questions provided that practical considerations

such as clear objectives, resource management, ap-

propriate sampling, and efficient analysis and

reporting are used effectively. Without those key con-

siderations, any sensory testing is likely to be mis-

guided and wasteful. The following examples have

been provided to illustrate how these common themes

apply to the different areas of sensory analysis.

Do You Want to Know if Consumers can

Distinguish Products?

0022 Consider the following hypothetical situation. You

own an icecream company. Your provider of sweet-

ener tells you to expect a substantial increase in price.

You do not want to make consumers pay more for

your icecream, so you find a different provider at a

lower price. Your biggest concern is providing the

same great flavor to your consumers. Therefore, an

initial question is: ‘If the new sweetener is used, can

people distinguish between the products?’

0023 There are numerous tests that can be used for this

purpose, collectively referred to as discrimination or

difference tests. Difference tests do not answer the

question ‘How much consumers like a product?’ or

‘How are these products different?’ Difference tests

answer one simple question, ‘Can people distinguish

samples?’

0024 Difference tests often test 30 or more known dis-

criminators in order to determine whether differences

that are found can be attributed to random variability

or are product-based. Both the analysis and reporting

for these tests are straightforward. Simple, test specific

tables (often based on binomial distributions) are

available to determine the likelihood that the differ-

ence between the samples is product-based. A concise

way to report these findings is to state the percentage of

people who could distinguish the products, followed

by the likelihood (a statistical p-value) that this differ-

ence can be attributed to random variability. For

example, results might be summarized as ‘In the paired

difference test, 51% (24 out of 47) selected the differ-

ent sample, which is less than the number required to

find a significant (p ¼0.05) difference in the products.

This suggests that consumers will not perceive ice-

cream with the new sweetener as different.’

Are You Interested in Exploring

Consumer’s perceptions?

0025 Again, let us imagine that you own the icecream com-

pany, but now you are interested in understanding

more about how your product is packaged. Your

main competitor uses different colors and a different

shape, and some consumers have noted that their

package is more ‘exciting’ and ‘appealing.’ You are

interested in knowing the specific characteristics of

the package that are related to how ‘eye-catching’ the

product is. What are the key factors – the color,

the material, the shape, the advertising on the

product?

0026Focus groups or one-on-one interviews are effect-

ive for initially exploring consumers’ perceptions.

This research is described as qualitative because the

objective is to answer the questions, ‘What is import-

ant to consumers and why?’, instead of quantifying

‘how much.’ Qualitative consumer research allows

for in-depth discussion of consumers’ attitudes and

perceptions.

0027As with any technique, the first step is using quali-

tative methods to define the specific objective. The

fundamental challenge of qualitative research is to

obtain relevant information from the consumer.

Given this, it is essential to develop a guide of

how the conversation (discussion) with the consumer

will proceed that will lead them to provide answers

to issues but will not bias their response. A well-

prepared guide is an important asset in managing

resources and obtaining the desired information

and is more than just an outline. It can provide

the interviewer or moderator with transitional

statements between topics, anticipate potential

problem areas, and provide ideas for ‘follow-up’

questions.

0028In qualitative consumer research, acknowledging

possible factors that may bias the sampling is import-

ant. Because the basic purpose of qualitative research

is to better understand consumers ideas, any influence

that causes an inconsistency between what consumers

communicate and what they actually believe may lead

to inaccurate conclusions being made. There are

many potential sources of bias, including: the facility

used, the attitude of the moderator, how questions

are asked, and the interaction among participants.

Effective focus groups, for example, generally in-

volve a comfortable, distraction-free room, with

the discussion led by a moderator, who promotes

an environment in which all participants feel com-

fortable expressing and discussing their ideas and

opinions.

0029Because of the nature of qualitative research,

analysis and reporting of findings are different than

with quantitative data. An effective way to analyze

qualitative data is to begin by reviewing all discussion

and finding ‘common themes’ that are related directly

to the objectives. For each theme, specific comments

from as many people as possible that illustrate these

SENSORY EVALUATION/Practical Considerations 5139

themes are noted. For reporting, the use of bulleted

points outlining the themes with some illustrative

quotes can be a concise, effective way to present

findings.

Do You Want to Estimate How Much

Consumers Like Products?

0030 Now, consider the situation: same icecream company,

only now you would like to sell more icecream. You

decide that a ‘fruity raspberry’ icecream is the best

new product because consumers like the flavor, and

your main competitor is having success with a similar

product. In short, you want a ‘fruity raspberry’ ice-

cream that consumers like more than your main com-

petitor’s. You produce a small amount of a new ‘fruity

raspberry’ icecream, but how do you determine how

much consumers like it?

0031 This objective requires a quantitative consumer

acceptance test. Consumer acceptance testing is a

generic phrase that describes a group of different

methods for estimating how much consumers like

products. In acceptance tests, consumers generally

evaluate multiple samples and are asked questions

including: overall liking and liking of basic product

characteristics (for example, sweetness, creaminess,

or raspberry flavor).

0032 One important key to using acceptance tests is

to ask questions that are directly related to the spe-

cific objectives of the test and to include the products

that are necessary. Because testing can be expensive,

clarity of objectives is important because cost con-

straints likely will limit the number of consumers who

can be tested or the number of questions that can be

asked. For example, in addition to overall liking, is it

important to obtain feedback on the overall flavor,

specific flavors, color, texture, aroma, meltdown?

Since one of the goals of this example is to determine

whether the product is liked more than the competi-

tor, samples that represent the competitors’ prod-

uct(s) are a prerequisite for testing.

0033 Keep the questions simple. Consumers generally do

not have the level of interest or attention that people

who work with the product every day have. For these

reasons, their willingness to think about confusing

issues, or respond to complex questions, may be

limited. If consumers do not understand the question,

or interpret a question in different ways, the results

are unlikely to be an accurate measure of the charac-

teristics that the questions were intended to answer.

0034 It is crucial that the participants in an acceptance

test are representative of the larger consumer popula-

tion. One of the best ways to ensure this is to test only

people who have been identified as potential con-

sumers of the products. In general, a consumer

acceptance test will collect responses from at least

100 consumers, but the actual number is dependent

largely on the decision being considered and may

range into the hundreds or, on rare occasions, into

the thousands.

0035Although the analysis of acceptance tests is more

complex than difference testing, the emphasis on ad-

dressing the specific objectives and brevity is no less

important. In this case, reporting the mean liking

scores and comparing the company’s new product

with that of the competition is usually the main area

of emphasis. If the new product does not score highly,

a review of the other consumer data is essential to

understand why.

Do You Want to Describe and Measure the

Sensory Properties of a Product?

0036Let us use the icecream company one last time. You

are receiving complaints that your vanilla icecream

has a ‘funny,’‘off,’ or ‘bad’ flavor. You attempt to

have consumers provide a more specific description

of the flavor, but they are unable to do this. That is

not surprising; consumers are great at communicating

what they like or dislike, but they are not as good at

describing why. Descriptive sensory analysis, using

highly trained assessors, is more appropriate for

this task because those assessors are able to reduce

complex perceptions that often involve multiple

senses into simpler senses that can be defined and

measured.

0037In our example, you have determined that the ice-

cream is close to, but not past, the ‘best if used by’

date. You are concerned that the shelf-life of the

product may be too long, but you need a better de-

scription of the flavor and its intensity during storage.

0038Product orientation and the use of appropriate

methods are important resource-management tools

in descriptive sensory testing. Allowing the panelists

to work briefly with products similar to what they

will test allows the panelists to become oriented with

the product and helps you and the panelists decide on

the best way to evaluate the products. Specific

methods and scales that are used for descriptive sens-

ory analysis are described extensively in the literature

and are dependent on the product being tested and the

objectives of the study.

0039Appropriate analysis and reporting for descriptive

sensory analysis again are dependent on the object-

ives. The objective of the sample study would be to

provide information on how the sensory properties of

the products are changing as storage time increases.

Thus, a chart that tracks changes in specific attributes

over time is a simple, effective method for evaluating

the changes.

5140 SENSORY EVALUATION/Practical Considerations

Conclusion

0040 Sensory analysis integrates many different sciences to

understand better the sensory properties of products

and consumer response to these properties. Because

sensory analysis has a broad range of applications,

start with the basic questions: ‘What do you want to

know?’, ‘What resources are available?’, ‘Who needs

to conduct the test?’, ‘What products will you use?’,

and finally, ‘What answers do you need?’ Answering

these questions will lead you to a consideration of the

practical aspects of conducting any sensory study.

See also: Sensory Evaluation: Food Acceptability and

Sensory Evaluation; Sensory Difference Testing; Sensory

Rating and Scoring Methods

Further Reading

Amerine MA, Pangborn RM and Roessler EB (1965)

Principles of Sensory Evaluation of Food. New York:

Academic Press.

Chambers E IV and Wolf MB (eds) (1996) Sensory Testing

Methods, 2nd edn. West Conshohocken, PA: ASTM.

Hootman RC (1992) Manual on Descriptive Analysis

Testing for Sensory Evaluation. Philadelphia, PA: ASTM.

Jellinek G (1985) Sensory Evaluation of Foods Theory and

Practice. Deerfield Beach, FL: Ellis Horwood.

Lawless HT and Klein BP (1991) Sensory Science Theory

and Applications in Foods. New York: Marcel Dekker.

Lyon DH, Francombe MA, Hasdell TA and Lawson K

(1992) Guidelines for Sensory Analysis in Food Product

Development and Quality Control. London: Chapman

& Hall.

Meilgaard M, Civille GV and Carr BT (1999) Sensory

Evaluation Techniques, 3rd edn. Boca Raton, FL: CRC

Press.

Moskowitz HR (ed.) (1988) Applied Sensory Analysis of

Foods, vol I and II. Boca Raton, FL: CRC Press.

Moskowitz HR (1983) Product Testing and Sensory Evalu-

ation of Foods Marketing and R&D Approaches. West-

port, CT: Food & Nutrition Press.

Munoz AM, Civille GV and Carr BT (1992) Sensory Evalu-

ation in Quality Control. New York: Van Nostrand

Reinhold.

Piggott JR (1988) Sensory Analysis of Foods, 2nd edn. New

York: Elsevier Science.

Poste LM, Mackie DA, Butler G and Larmond E (1991)

Laboratory Methods for Sensory Analysis of Food.

Ottawa: Canada Communication Group-Publishing

Centre.

Stone H and Sidel JL (1993) Sensory Evaluation Practices,

2nd edn. San Diego, CA: Academic Press.

Watts BM, Ylimaki GL, Jeffery LE and Elias LG (1989)

Basic Sensory Methods for Food Evaluation. Ottawa:

The International Research Centre.

Yantis JE (ed.) (1992) The Role of Sensory Analysis in

Quality Control, Manual 14. Philadelphia, PA: Ameri-

can Society for Testing and Materials.

Sensory Difference Testing

B Rousseau, The Institute for Perception, Davis,

CA, USA

Introduction

0001This article describes our current knowledge on the

topic of sensory discrimination testing. The general

objectives of the commonly called ‘difference tests’

are first emphasized, the main objective being the

scaling of sensory magnitudes. Then, the traditionally

encountered discrimination protocols are described in

terms of the nature of the instructions given to the

subject. The common data analysis based on bino-

mial tables and its limitations are mentioned. An

approach based on an index of perceptual differences,

called d

and pronounced ‘d prime,’ is introduced. It is

shown how the results using this index are consistent

across methods. A comparison of the power of the

traditional discrimination tests is presented, showing

which tests are less likely to fail to detect nonnegligi-

ble sensory differences between food products.

0002The field of sensory evaluation has been commonly

divided into three categories: sensory discrimination

testing, descriptive analysis, and hedonic studies. The

sensory discrimination tests are traditionally used to

investigate whether two food samples can be per-

ceived as significantly different by a group of panel-

ists. Descriptive analysis will provide qualitative

information regarding the difference among food

samples, by focusing on particular sensory attributes

inherent to the products under study. Hedonic testing

will be used in order to study the acceptance and

preference of food samples by consumers.

0003Discrimination testing is sometimes considered as a

straightforward data collection method, with results

providing useful but limited information. One will

use a difference test in order to determine whether

the sensory characteristics of two samples, for in-

stance a standard product and its reformulation, can

be distinguished. Assuming that a proper experimen-

tal design is used, the similarity or difference of the

two products will be concluded. However, this

reasoning can appear unsatisfactory. It could be easily

argued that two products with different physical and

chemical characteristic will never be perceived as

identical. If a large enough sample size is used, any

sensory difference will be detected.

0004This brings us to the notion which will be used in

the present article. Instead of merely answering the

question: ‘Are my products the same or different?,’

we will show how discrimination test results can be

SENSORY EVALUATION/Sensory Difference Testing 5141

used as a tool to answer the following question: ‘How

large is the sensory difference between my products?’

These results will permit the scaling of sensory mag-

nitudes and sensory differences, information thought

until now to be provided only through the use of

rating methods.

Traditional Protocols Used in

Discrimination Testing

0005 There is no limitation on the number of different

discrimination tests that can be used to study

sensory differences. The list which will be given here

is not exhaustive, and alternative discrimination

methods can easily be created. The issue is then

to develop the proper model to conduct statistical

analyses.

0006 The discrimination protocols can be classified into

two main categories: those that require the nature of

the sensory difference to be specified in the instruc-

tions and those that do not.

0007 The protocols described here will be illustrated by

the comparison of two cookies A and B, B being

slightly sweeter than A. The samples are presented

blind, and no other difference (visual, textural, tem-

perature) is assumed to be present. A subject will

perform several tests in succession, or a group of

subjects will perform one test each. From the results

of the test, the similarity of the difference between the

two samples will be concluded with a given level of

confidence, depending on the sample size (number

of tests) involved in the comparison.

Directional Difference Tests

0008These methods require the nature of the difference to

be specified in the instructions. Table 1 gives details

for each method.

0009The chance probability represents the probability

of getting the answer right if the subject cannot dis-

criminate between the products. For the sorting

protocols, it is important to keep in mind the

fatiguing nature of food samples and the memory

requirements that might significantly hinder subjects’

performance. These tasks are best suited to textural

and visual investigations.

Nondirectional Difference Tests

0010Those are the most commonly used methods in sens-

ory discrimination testing, even if their statistical

power can be up to 100 times inferior to that of the

directional difference tests (Table 2).

Traditional Data Analyses

0011The traditional way of analyzing data extracted from

difference tests involves no consideration of the psy-

chological processes occurring at the time a response

is generated. This is what is called a response-based

analysis. The information obtained from this rather

limited statistical analysis is either ‘Yes, the products

are different,’ or ‘No, they are not different.’ No

information is obtained regarding the size of the sens-

ory difference. This can yield inconsistent results and

this issue will be discussed later.

tbl0001 Table 1 Directional difference tests

Test Number of samples Instructions Chance probability

2-AFC Pairs of samples: A:1,B:1

Find the sweeter cookie

1/2

3-AFC Triads of samples: A:2,B:1orA:1,B:2

Find the sweetest cookie

Find the least sweet cookie

1/3

A/Not A Single sample: A or B

Is it A or is it Not A (B)?

1/2

Sorting n samples: A: np, B: p Find the p sweetest samples

p!ðn  pÞ!

AFC; alternative forced choice.

5142 SENSORY EVALUATION/Sensory Difference Testing

0012 In discrimination testing, one is concerned with

two types of error. The first one is the type I error:

concluding that the products were significantly differ-

ent while they were not. The second is the type II

error: failing to detect an existing sensory difference.

Statistical tests are conducted at a particular a level,

usually 0.05, describing the type I error. An a level of

0.05 means that we have only a 5% chance of con-

cluding that the products were different while they

were not. For the type II error, described by the

parameter b, the level used is usually 0.2 or 0.1.

The statistical power of the test is defined by:

Power ¼1b. A power of 80% implies that a test

has 80% chance of detecting a difference of a given

size. The issue of power will be discussed later.

Binomial Analysis

0013 For a given protocol, the number of correct tests is

compared to the number that would be expected by

chance. This type of model is called a guessing model.

It assumes that if the subject cannot discriminate

between the samples, he or she will pick a sample or

an answer randomly. We are testing the H

hypothesis

that the two products are identical. If rejected, we will

accept the H

hypothesis that the products are differ-

ent. For instance, if a study with 20 tests was run

using the two-alternative forced choice (2-AFC)

protocol and 14 were performed correctly, this result

would be compared to an expected chance result of

10. Tables have been published indicating the min-

imum number of tests correct for a given sample size

to be significant at a given a level, usually 0.05. In our

example, to be significant at the 5% level, we would

need 15 tests correct. Therefore, we cannot conclude

that the two cookies were significantly different. We

cannot reject H

0014It is critical to be aware that we are not concluding

that the cookies are the same. We can only conclude

that they are not different. Had we used a larger

sample size, we might have obtained a significant

result. In order to accept H

, it is necessary to

consider the power of the test.

Analysis

0015This analysis is traditionally used with tests yielding

results in a contingency table, such as the A/Not A

and same–different tests. An example of a contin-

gency table is shown in Table 3. A study between

tbl0002 Table 2 Nondirectional difference tests

Test Number of samples Instructions Chance probability

Triangle Triads of samples: A:2,B:1or

A:1,B:2

Find the different sample

1/3

Duo–trio Triads of samples:

Reference: A and A:1,B:1or

Reference: B and A:1,B:1

Find the sample more similar

to the reference

1/2

Same–different Pairs of sample: AA, BB,orAB

Are the samples the same or

different?

1/2

p-out-of-nnsamples: A: n  p, B: p Make one group of n p similar

samples and one group of p

similar samples

If n is odd :

p! ðnpÞ!

If n is even : 2 

p! ðnpÞ!

SENSORY EVALUATION/Sensory Difference Testing 5143

the two cookies A and B was conducted with 100

consumers, each getting either a different pair (AB or

BA) or an identical pair (AA or BB).

0016 A w

value can be calculated and we find that

¼9.1. Looking in a table for the significance of

, we find that a w

of 9.1 and 1 degree of freedom

is significant at the 1% level.

0017 Therefore, here we can reject the null hypothesis

and conclude that the products are different.

Taking into Account the Degree of

Difference Between the Products:

Thurstonian Models

0018 The data analyses discussed above only provide one

type of information from discrimination testing: ‘Are

my products significantly different or not?’ We never

mentioned the magnitude of the difference between

the products. Was it a large difference or a small

difference? How does it compare to results from pre-

vious studies? In order to answer these questions, it is

necessary to introduce new notions, such as an index

of perceptual difference, called d

. We are first going

to show why it is essential to consider this index when

comparing products.

Insufficiency of the Guessing Model: The Paradox

of Discriminatory Nondiscriminators

0019 It has been reported numerous times in the literature

that, while using the same subjects and the same

products, the null hypothesis could be rejected using

some protocols but not others. The most illustrious

example is the result discrepancy between the 3-AFC

and triangle tests. While the two tests have the same

design and only very slightly different instructions,

subjects’ performance is significantly greater in the

3-AFC than in the triangle test. With a 3-AFC test, it

might be concluded that the products are significantly

different, while we would not be able to reach that

conclusion with the triangle test. Which result is

right? Or are the two methods giving two versions

of the same information, i.e., the magnitude of the

difference between the two products? With the tri-

angle test, nonnegligible sensory differences might

not be detected because of insufficient statistical

power. This might result in the release of reformulated

products in the marketplace that will be rejected by

consumers.

0020These experimental results underscore the insuffi-

ciency of relying on the number of correct answers in

a given test to determine how different two products

are. It is essential to take into consideration two very

important aspects of sensory evaluation: variability

and behavior. These aspects are taken into account in

Thurstonian modeling.

Thurstonian Models

0021These models are very similar to those encountered in

the field of psychology and discussed in signal detec-

tion theory.

0022Thurstonian modeling assumptions

Variability Let’s take our sweet cookie example

again. The idea behind perception variability is that,

when tasting the same cookie several times, the per-

ceived sweetness will not always be the same. On

average it will have a certain intensity, but at any

given moment, the perceived intensity can be slightly

stronger or slightly weaker (Figure 1). Several reasons

can explain these variations: first, there is always

some random noise in the nervous system due to the

spontaneous firing of nerves. Second, the number of

compounds binding with the sweet receptors can

be slightly different each time. Third, there might be

variation in the cookie itself: the sweet compounds

might not be evenly distributed on the cookie, or the

subject might take bites of slightly different sizes

every time.

0023The likelihood of each sweetness magnitude occur-

ring for the cookie can be represented by a distribu-

tion. This distribution is usually assumed to be

normal, even though alternative models have been

published using other types of distributions. Figure 2

illustrates this concept. The height of the distribution

at each intensity level represents the likelihood of this

intensity occurring in a given tasting. The intensity at

the mean of the distribution is the most likely. The

further away the intensity is from the mean, higher or

lower, the less likely it is to occur.

tbl0003 Table 3 Results of a same–different test conducted with 100

consumers

Presentation Response

‘Different’ ‘Same’

Same 20 30

Different 35 15

Sweetness intensity

StrongerMeanWeaker

fig0001Figure 1 Representation of variable sweetness intensities for a

cookie.

5144 SENSORY EVALUATION/Sensory Difference Testing

0024 When performing a discrimination test, the two

products to be compared can be represented by two

normal distributions. This is shown in Figure 3.

0025 In the top picture, product Y is always sweeter than

product X. In this situation test, a subject will always

get 100% tests correct in a difference test. Discrimin-

ation testing is not even necessary in this case; X and

Y are totally discriminable. In the bottom picture,

there is some overlap between the two distributions.

Overall, B is sweeter than A and will be perceived as

such most of the time, but on some occasions, B will

be perceived as less sweet than A. This will happen

when the B intensity is on the lower part of its distri-

bution and the A intensity is on the higher part of the

distribution. The measure of the extent of the overlap

is called d

and is the distance between the means

of the two distributions measured in terms of their

standard deviation. A d

of 1 means that the two

distributions are 1 standard deviation apart. The

bigger the d

, the more different the products. A d

of 0 means that the products are undistinguishable.

N.B.: a d

of 1 corresponds to 76% tests correct in a

2-AFC test. In psychophysics, this would correspond

to the definition of a detection threshold.

0026 We now have an index measuring the magnitude of

a difference, something we did not have when using

the guessing model. Let’s see now how we can obtain

a d

value when using discrimination tests. For this,

we need to take into account the behavioral aspect

of discrimination testing, namely the decision rule, or

cognitive strategy, used by the subject when perform-

ing a discrimination test.

0027Behavior The guessing model assumes that subjects

make mistakes when they cannot tell the difference

between the products and make a wrong guess about

the answer. We will see that this is not how incorrect

answers arise.

0028Duo–trio example Let’s imagine that a subject per-

forms a duo–trio test between two cookies A and B,

the less sweet cookie, A, being the reference. A trial is

described in Figure 4.

0029In the top picture, the subject tasted the reference

and perceived a sweetness on the left-hand side of

the distribution (a

). He then tasted the two alterna-

tive samples and perceived the a and b sensations.

When asked which of the two is more similar to the

reference, he will choose a, since it is closer to a

. His

answer will be correct.

0030In the bottom picture, we can see that the sensa-

tions perceived will yield an incorrect answer. This

time, b is closer to a

than a is, thus the subject will

pick b as more similar and his answer will be wrong,

even if he gave the right answer based on the sensa-

tions perceived. We see that the error was not

made because the subject guessed, but because the

perceived sensations happened to be in such locations

that a wrong answer was generated.

0031Over repeated trials, the top situation will occur

more often than the bottom one, resulting in the

measure of a d

value and the conclusion that the

products are different.

0032The d

values can easily be calculated from discrim-

ination test results. Tables have been published that

provide the correspondence between d

and propor-

tion of correct answers for the most traditionally used

protocols.

0033Main cognitive strategies For each discrimination

protocol, a subject will use a specific cognitive strat-

egy. The two main cognitive strategies are the com-

parison of distances strategy and skimming strategy.

Figure 5 illustrates these two decision rules with the

triangle and 3-AFC tests.

0034Figure 5 represents a triadic test where two A and

one B samples were presented. In the triangle test, the

Weaker

Likelihood

Sweetness intensity

Mean Stronger

fig0002 Figure 2 Normal representation of the likelihood of sweetness

perception.

Sweetness intensity

d

fig0003 Figure 3 Thurstonian representation of two products differing

in sweetness.

Sweetness intensity

Incorrect

Correct

Sweetness intensity

fig0004Figure 4 Correct and incorrect answers in a duo–trio test.

Square, reference; circle, alternative sample.

SENSORY EVALUATION/Sensory Difference Testing 5145

subject will compare the distances to determine which

of the three samples is most different from the other

two. We see that the subject will select the A sample

furthest left and his answer will be recorded as incor-

rect. For the 3-AFC, the subject needs only to select

the sample with the largest magnitude. Therefore

he will pick the B sample (most intense) and his

answer will be recorded as correct. This example

illustrates why subjects’ performance will be lower

in the triangle than in the 3-AFC test.

0035 We can generalize this example to the other proto-

cols described earlier. For directional discrimination

tests (m-AFC tests), subjects will use a skimming

strategy. For the nondirectional protocols (triangle,

duo–trio), subjects will use a comparison of distances

strategy. The A/Not A and same–different tests use

alternative strategies, since the instructions require

the use of a psychological criterion (How strong

does a sample need to taste before I call it ‘A’? How

different do two samples need to taste before I call

them ‘Different’?) The A/Not A test has a strategy

analogous to a skimming strategy, while the same–

different test has a strategy analogous to a compari-

son of distances strategy.

0036 Protocols with a skimming strategy are statistically

more powerful, i.e., not as likely to fail to detect a

nonnegligible difference as a test using a comparison

of distances. This was illustrated earlier with the

triangle and 3-AFC tests.

0037 When using d

values, results from discrimination

tests now become independent of the protocol used.

This is what we would predict, as discrimination

tests are merely an instrument to measure product

differences. Simply calculating the proportion of

tests correct and using the binomial test will provide

bias and unstable information.

0038 Practical aspects of Thurstonian modeling Now

that we have ways of measuring a degree of difference

between food products, we need to set up the experi-

mental parameters that will allow us to detect it

accurately. This brings us to the issue of power.

0039 Power In order to define the power of a test, we

need to decide what degree of difference we want to

be able to detect. This will require some general

knowledge of the magnitude of the difference that

usually needs to be detected and can be determined

using past data and converting them in terms of d

values. The power of a test can be illustrated by

calculating the sample size needed to detect a given

with a given confidence. The smaller the sample

size required, the more powerful the test. Equations

have been published providing the sample size needed

for a given d

value, a level, and power. Tables can be

constructed and an example is shown in Table 4.

0040Here is the way to read the table. For the 2-AFC,

first line: if we run 78 tests and detect a significant

difference at the 5% level, we are 95% sure that

a difference exists between the products. If we do

not detect a significant difference at the 5% level,

we are 80% sure that the d

was less than 0.5. In

order to reach the same confidence level for the

duo–trio, 3-AFC, and triangle tests, we need a sample

size of 3092, 64, and 2742 respectively. These vari-

ations in sample size indicate the larger power of the

protocols using the skimming strategy.

0041The same–different and A/Not A tests cannot be

included in this table because of the issue of response

bias. A power value would need to be calculated for

each criterion location and size. However, as a gen-

eral rule, the same–different test is slightly more

powerful than the triangle and duo–trio tests but

less powerful than the 2- and 3-AFC tests, while the

power of the A/Not A test is closer to that of the

2-AFC test. N.B.: power tables have been published

using the proportion of discriminators in the popula-

tion. This notion is flawed (it is directly linked to the

proportion of correct answers, which we’ve seen is

not a reliable measure of the degree of difference

between products) and such tables should be used

with extreme caution.

Triangle

Comparison

of distances

3-AFC

Skimming

fig0005 Figure 5 Comparison of distances and skimming strategies in

the triangle and three-alternative forced choice (3-AFC) tests.

tbl0004Table 4 Sample size required to detect a given degree of

difference (d

) for given a and power levels and discrimination

protocol

a Power 2-AFC Duo^trio 3 -AFC Triangle

0.5 5% 0.8 78 3092 64 2742

1.0 5% 0.8 20 225 15 197

1.5 5% 0.8 9 55 6 47

0.5 1% 0.8 128 5021 103 4441

1.0 1% 0.8 34 366 25 318

1.5 1% 0.8 16 91 11 76

0.5 5% 0.9 108 4283 89 3810

1.0 5% 0.9 27 310 21 276

1.5 5% 0.9 12 76 9 66

0.5 1% 0.9 165 6511 135 5776

1.0 1% 0.9 43 473 32 416

1.5 1% 0.9 20 117 14 100

AFC, alternative forced choice.

5146 SENSORY EVALUATION/Sensory Difference Testing