Dubois E., Gray P., Nigay L. (Eds.) The Engineering of Mixed Reality Systems
Подождите немного. Документ загружается.
136 E.T. Khoo et al.
Fig. 7.5 Floor square lighted up when a player steps on it
the older players noted that when they stepped on a square, the light for that square
lit up, as shown in Fig. 7.5. This gives immediate feedback to the users that their
action is being registered by the s ystem and it reinforces that they understood the
purpose of t his feature.
Eighty percent of the young players gave feedback that the handheld controller
should have haptic feedback for the game events, for example, the reduction of
an energy level. They explained that due to many tasks that they need to perform
during the game, feedback that does not require their active awareness, for example
vibration feedback, is much preferred. The group of older people did not have this
concern. They may not have had exposure to this type of multi-modal experience in
a game setting previously.
Each player wore a lanyard-style wearable LED display, which hung around their
neck that lit up during game events, for example, when hit by a laser, collection of
bonus items. None of the users in the study looked at their own display. We found
out that it is not intuitive nor easy physically to have the users look at the display
while playing the game. One suggestion is to integrate the display into the handheld
controller. For future improvement, we will incorporate vibration feedback and an
array of multicolor LEDs on the handheld controller as a form of feedback of game
status and events.
7 Mixed Reality Intergenerational Entertainment System 137
7.6.4 Physicality Issues of the Virtual and Physical Player Roles
The Age Invaders platform involves the players in a physical game space arena
focused on a room-sized game board with 45 tiles each representing a physical posi-
tion the players can occupy. Physical players register their position in one square at
a time since the size of the square is just large enough for one person standing in
place. While occupying a square, it is reserved for their avatar in the virtual world
enforced by the game logic.
The presence of the virtual player is felt on the physical game space by the evi-
dence of their activities in the virtual game space including the placement of barriers
and hearts or by changing the speed of the l aser beams. The virtual player’s activi-
ties were not easily understood by the physical players as being activity directed by
the human player versus coming from the logic of the computer program.
We have identified a footprint timer range of 1–3 s to be the most exciting for the
young players. Most young players felt the game was challenging because they have
to take care of a few tasks at the same time, for instance, following the step patterns,
avoiding the approaching laser beams, and trying to launch one. We observed that
the footprint makes the game physically more interesting and challenging to the
young and less demanding for the older people as they have the freedom to move to
any position as they do not need to follow any sequence of steps. This matches the
physical effort to the desired amount of movement that the adult prefers. Through
the nature of the team competition, the adult is compelled to move into a position to
take offensive action, but can easily reduce the amount of movement and play from
a more defensive style.
Most players who had experienced the game play from the virtual player role
expressed that they found the game play experience less enjoyable than the physical
player role. This could have been for a number of reasons which might include the
following:
– The physical players interact in a more social situation in which they see the other
players
– Interaction in the physical space involves natural conversation and body language
– Interaction in the physical space adds the element of performing onstage for
others, while the virtual player does not receive the same feedback
– The physical exertion of the virtual player is much less and involves small hand
movements with the mouse and keyboard.
A possible avenue for future development could be to represent the virtual
player’s avatar in the physical space and participate as a player at the same level.
This could allow for a virtual player that has to dodge laser beams and follow dance
steps just like the physical player, as opposed to acting as a referee or just balanc-
ing the game play. This could be realized with a robot in the physical game space,
which the virtual player could control. In place of a robot, a virtual character could
be used which could be represented as a special avatar on the physical and virtual
game space.
138 E.T. Khoo et al.
On the other hand, the interface for the virtual player could be modified to include
a more physical interaction as well. For example, a DDR-style dance mat could be
used to give the physical exertion component to the virtual player who is partici-
pating via the Internet and located far from the physical game space. However this
feature may not be suitable for parents who are at work.
7.6.5 Focus Group Session with Older Players
Five weeks after we conducted the initial user study for t he older and younger peo-
ple, we went back to the senior center to conduct a follow-up focus group session
with seven of the older players as shown in Fig. 7.6. Our aim was to investigate
the longer lasting impact of the Age Invaders game. When prompted to describe
the most memorable aspects about the game, all of them mentioned the launching
of lasers, avoiding lasers, and chasing after the hearts on the game board. We iden-
tified that for the older players, the memorable actions in the game are physical
in nature. The player has to press the physical button on the wireless controller to
launch the virtual laser and in the cases of avoiding lasers and chasing after the
hearts, t he players have to physically move their bodies.
“This is a fresh idea and it was my first time playing” quoted by one of the older
people, to which all agreed. They were all excited about this new entertainment
platform. The older people have never performed any computer-related activities or
played electronic games before and are afraid to attempt them. However, the play-
ers see this physical game system as being so different from traditional computer
systems that not only were they comfortable to use the system but they were also
having fun interacting with other players while playing.
When asked what skills were involved in playing the game and about the diffi-
culty level, all agreed that the game was easy to learn and that the game speed was
Fig. 7.6 Focus group session with seven older players
7 Mixed Reality Intergenerational Entertainment System 139
fine. A few mentioned that the pace could be a little slower in order to play for a
longer period of time. However, they emphasized that the game speed cannot be
too slow, otherwise it would become boring. The current game pace is exciting for
them. An interesting description was stated by one of the players, “The game gives
me good exercise. When you see a heart, you have to run to collect it. If a laser
is approaching, you have to avoid it quickly. The game is very engaging and can
we play it again?” This supports the design goal of doubling the game as a form of
exercise. We also noticed that the older people enjoy moderate physical activity that
challenges their visual–motor coordination.
When asked about how much fun was the game as played, they all gave the
maximum rating. “Thinking about it now, we can’t wait to play again!” said one
of them. It is obvious that the older players have enjoyed the game and have high
motivation to play the game again. Of particular interest is the fact that the users do
not see a strong connection between this system and traditional electronic gaming
and computing in general. This high motivation level is valuable in and of itself as
confirmation that older adults can be engaged in new technologies and have high
enjoyment levels. Ongoing research is underway to understand the long-term use of
the system.
7.7 Conclusion
We have presented our user-centered design process for intergenerational family
entertainment. This chapter describes the design issues and decisions made dur-
ing the development of a system with the goal of promoting social and physical
interaction across the generations. We utilized the user-centered design approach
of keeping the needs of the user central to the project development and constantly
validating our assumptions with testing of incremental changes and later stage test-
ing involving game sessions closer to the intended context of use. By following this
method, we have been able to create a system, which not only meets the user needs
for physical interactive entertainment but has also served to better understand user
interaction in the mixed reality context. The extensive user studies helped to shape
and construct the physical dimensions of the system beginning in the earlier stages
including setting the size of the game squares and the handheld control device. Later
refinements guided by the users included refinements to the virtual character role
and also highlighted the key strengths of the game scenario. It was precisely the
physical nature of the game system that presented the older players with a digital
game experience they could easily understand. The older players, who normally
express concern and apprehension about computers, were able to enjoy the interac-
tive system and were anxious to try new games made for the platform. The social
aspects of the game play were appreciated by two distinctly different groups of play-
ers who would not normally play together. Through the variable game parameters,
the effort level for all players was set according to physical abilities, which ensured
that all players were challenged physically, yet still fostered collaborative compet-
itive play. According to our findings, when making design decisions in developing
140 E.T. Khoo et al.
intergenerational family entertainment systems, the use of one portion of the user
base, which is most proficient with technology before involving the novice users,
should be considered. During the user studies, we noticed that the children assisted
the older players to verbalize the situation and difficulties that they were facing with
the game. This was found in our research to empower the novice users to accept
digital t echnology and increased the playability of the game.
References
1. National Research Council USA. The aging population in the twenty-first century. 1998.
Virtual Reality, 12(1):3–16, March 2008.
2. Bucar A and Kwon S. Computer hardware and software interfaces: Why the elderly are under-
represented as computer users. CyberPsychology & Behavior, 2(6):535–543, December 1999.
3. Thomas KGF, Laurance HE, Luczak SE, and Jacobs WJ. Age-related changes in a human
cognitive mapping system: Data from a computer-generated environment. CyberPsychology
& Behavior, 2(6):545–566, December 1999.
4. Shneiderman B. Designing the user interface: Strategies for effective human-computer
interaction. Addison-Wesley, New York 1998.
5. Jaycox K and Hicks B. Elders, students, and computers. University of Illinois, 1976.
6. Lazzaro N. Why we play games: Four keys to more emotion without story, March 8, 2004.
http://www.xeodesign.com/whyweplaygames.html.
7. Granny got game? http://whatjapanthinks.com/2008/02/26/granny-got-game/.
8. Cheok AD, Goh K, Liu W, Farbiz F, Fong S, Teo S, Li Y, and Yang X. Human pacman: A
mobile, wide-area entertainment system based on physical, social, and ubiquitous computing.
Personal and Ubiquitous Computing, 8(2):71–81, May 2004.
9. Khoo ET, Cheok AD, Nguyen T, and Pan Z. Age invaders: Social and physical inter-
generational mixed reality family e ntertainment. Virtual Reality, 12(1):3–16, March 2008.
10. Khoo ET, Merritt T, Cheok AD, Lian M, and Yeo K. Age invaders: User studies of inter-
generational computer entertainment. Entertainment Computing ICEC 2007, 4740:231–242,
2007.
11. Christian H. An activity analysis of electronic game simulators. Therapeutic Recreation
Journal, 12:21–25, 1978.
12. McGuire F. Improving the quality of life for residents of long term care facilities through
videogames. Activities, Adaptation and Aging, 6:1–7, 1984.
13. Weisman S. Computer games and the frail elderly. Gerontologist, 23:361–363, 1983.
14. Whitcomb GR. Computer games for the elderly. SIGCAS Computers and Society,
20(3):112–115, 1990.
15. Matsuo M, Nagasawa J, Yoshino A, Hiramatsu K, and Kurashiki K. Effects of activity
participation of the elderly on quality of life. Yonago Acta Medica, 17(46):17–24, 2003.
16. Babyak M, Blumenthal J, Herman S, Khatri P, Doraiswany M, Moore K, Craighead W,
Baldewicz T, and Krishnan K. Exercise treatment for major depression: Maintenance of
therapeutic benefit at 10 months. Psychosomatic Medicine, 62(5):633–638, 2000.
17. Bogost I. The rhetoric of exergaming. In Proceedings of the Digital Arts and Cultures (DAC)
Conference 2005, December 2005.
18. Fuyuno I. Brain craze. Nature, 447:18–20, 2007.
19. Wii. http://en.wikipedia.org/wiki/Wii.
20. Gronbaek K, Iversen OS, Kortbek KJ, Nielsen KR, and Aagaard L. Igamefloor: A platform
for co-located collaborative games. In ACE ’07: Proceedings of the international conference
on Advances in computer entertainment technology, pages 64–71, New York, NY, USA, 2007.
ACM.
7 Mixed Reality Intergenerational Entertainment System 141
21. Mueller F, Agamanolis S, and Picard R. E xertion interfaces: Sports over a distance for social
bonding and fun. In CHI ’03: Proceedings of the SIGCHI conference on Human factors in
computing systems, pages 561–568, New York, NY, USA, 2003. ACM.
22. Florian Mueller and Stefan Agamanolis. Exertion interfaces. In CHI ’07: CHI ’07 extended
abstracts on Human factors in computing systems, pages 2857–2860, New York, NY, USA,
2007. ACM.
23. Florian Mueller and Stefan Agamanolis. Exertion interfaces. In CHI ’08: CHI ’08 extended
abstracts on Human factors in computing systems, pages 3957–3960, New York, NY, USA,
2008. ACM.
24. Lena Mamykina, Elizabeth D. Mynatt, and David R. Kaufman. Investigating health man-
agement practices of individuals with diabetes. In CHI ’06: Proceedings of the SIGCHI
conference on Human Factors in computing systems, pages 927–936, New York, NY, USA,
2006. ACM Press.
25. Boehm BW. A spiral model of software development and enhancement. Computer,
21(5):61–72, May 1988.
26. Fallman D. Design-oriented human-computer interaction. In CHI ’03: Proceedings of the
SIGCHI conference on Human factors in computing systems, pages 225–232, New York, NY,
USA, 2003. ACM Press.
27. Gulliksen J, Göransson B, Boivie I, Blomkvist S, Persson J, and Cajander Å. Key principles
for user-centred systems design. Behaviour & Information Technology, 22(6):397–409, 2003.
28. State of the family in Singapore. http://fcd.ecitizen.gov.sg/CommitteeontheFamily/Research/
Executive+Summary.htm.
29. Human factors/ergonomics handbook for the design for ease of m aintenance.
http://tis.eh.doe.gov/techstds/standard/hdbk1140/hdbk1140.html.
30. Sweetser P and Wyeth P. Gameflow: A model for evaluating player enjoyment in games.
Computers in Entertainment (CIE), 3(3):3, July 2005.
Chapter 8
Auditory-Induced Presence in Mixed Reality
Environments and Related Technology
Pontus Larsson, Aleksander Väljamäe, Daniel Västfjäll,
Ana Tajadura-Jiménez, and Mendel Kleiner
Abstract Presence, the “perceptual illusion of non-mediation,” is often a central
goal in mediated and mixed environments, and sound is believed to be crucial
for inducing high-presence experiences. This chapter provides a review of the
state of the art within presence research related to auditory environments. Various
sound parameters such as externalization and spaciousness and consistency within
and across modalities are discussed in relation to their presence-inducing effects.
Moreover, these parameters are related to the use of audio in mixed realities and
example applications are discussed. Finally, we give an account of the technological
possibilities and challenges within the area of presence-inducing sound rendering
and presentation for mixed realities and outline future research aims.
Keywords Presence · Auditory · Auralization · Sound · Acoustics · Virtual
environments · Mixed reality · Augmented reality
8.1 Audio in Mixed Realities
As with virtual reality (VR) and virtual environment (VE), the concept of mixed
reality (MR) has come to mean technologies that primarily concern various forms
of visual displays. There is nonetheless a range of possibilities of using audio for the
purpose of mixing realities and blurring the boundary between what is real and what
is not. The purpose of integrating audio into the MR application is usually not only
to add functionalities and alternative information presentation, but also to enhance
the user experience.
P. Larsson (B)
Volvo Technology Corporation, SE-405 08 Göteborg, Sweden
e-mail: pontus.larsson@chalmers.se
143
E. Dubois et al. (eds.), The Engineering of Mixed Reality Systems, Human-Computer
Interaction Series, DOI 10.1007/978-1-84882-733-2_8,
C
Springer-Verlag London Limited 2010
144 P. Larsson et al.
A great advantage of audio compared to visual displays is that the mediation
technology may be less visible and less obvious – for example, when using hid-
den loudspeakers or comfortable, open headphones. Compared to visual displays,
mixing virtual and real audio is also relatively easy and cost-efficient: simply super-
imposing virtual sound on the real sonic environment using open headphones is
usually sufficient for many applications (a technique comparable to optical mix-
ing in visual displays) [1]. Using closed headphones with external microphones
attached to mix virtual and real digitally is also possible and has been subject to
some recent research [2]. This technique has the advantage of giving the possibility
of efficiently attenuating certain sounds and letting others through but comes with a
higher computational demand.
The extent to which audio is used in MR may of course vary and one may
use audio for several different purposes, ranging from full-blown, entirely vir-
tual 3D soundscapes with hundreds of sound sources and realistic room acoustic
rendering seamlessly blended with the real-world acoustics, to very simple mono-
phonic sounds produced via one loudspeaker (cf. Apple’s Mighty Mouse
TM
, which
has a built-in piezoelectric loudspeaker producing click sounds). Different audio
rendering techniques may of course also be combined in the same application.
It is rather straightforward to define different levels of reality mixing when talk-
ing about entirely visual stimuli/displays. One may distinguish four different distinct
visual display cases along the virtuality continuum [3]: real environment (RE), aug-
mented reality (AR), augmented virtuality (AV), and virtual environment (VE). The
same cases may of course also be identified for auditory displays. Cases explaining
auditory RE and VE are probably superfluous, but an AR audio application may for
example be an interactive museum guide telling stories and augmenting sound to the
different objects and artifacts as the museum visitor approaches them. Audio inte-
grated in interaction devices and in such way increases the impression of the devices
being a part of the mediation system could also be seen as audio AR. An example
of using audio in interaction devices is the Nintendo Wii
TM
remote, which actually
has a built-in loudspeaker which produces sound when the user, e.g., hits a virtual
tennis ball or swings a virtual golf club. This audio augmentation thus provides a
connection between the interaction device and the visual world – efficiently mixing
the real and the virtual also across the displays.
The AV case, where a real object is mixed with the VE, is perhaps less dis-
tinguishable within the realm of auditory displays but could be exemplified by,
e.g., systems where the user’s own voice is integrated with the VE [4]. Another
example is when the user’s footsteps are amplified and presented within the VE
to provide interaction feedback and enhance presence (see [5], although this study
used synthetic generation of the footstep sounds).
In many cases when we discuss MR/VE audio, however, the sound is also accom-
panied by a visual display or visual information in some other form. For example,
in our museum guide case above, the visitor not only hears the augmented audio but
naturally also watches all the museum artifacts, reads signs and displays, and expe-
riences the interior architecture visually. In more advanced systems, the auditory
virtual information is also accompanied by digital visual information of some degree
of virtuality. As such, the MR situation – or at least the taxonomy – becomes more
8 Mixed Reality Environments and Related Technology 145
complex; for audiovisual displays, there should exist at least 16 different distinct
ways of mixing realities as shown in the table below.
As displayed in the table, the only cases which are not MR are the ones where
we have real or virtual displays in both modalities at the same time, i.e., RE/RE
and VE/VE – between these end points we have a continuous range of audiovisual
reality mixing possibilities. It should be noted however that to be defined strictly as
MR it seems reasonable that for cases where reality in one display is mixed with
virtual information in another display, the content provided by reality should be
relevant to the virtual environment. Thus, when we have a VE visual display and
a real auditory environment (top right cell in Table 8.1), the auditory environment
should contain information that is relevant to the visual environment for the system
to be defined as an MR system. As an example, disturbing noise from projecting
systems can probably not be classified as a relevant sound for the visual VE while
the voice of a co-user of the VE most likely is a relevant and useful sound.
Regardless of the type of (audiovisual) reality mixing, the central goal of many
MR systems is, or at least should be, to maximize the user experience. However,
with added sensory stimulation and system complexity, measuring the performance
of MR in terms of user experience is becoming more and more difficult. For this
reason, the concept of presence (the user’s perceptual illusion of non-mediation [6])
is important since it offers a common way of analyzing and evaluating a range of
media technologies and content, regardless of complexity. Obtaining presence is
also often a central goal of many mediation technologies such VR, computer games,
and computer-supported communication and collaboration systems, and, as we will
explain in the next section, sound and auditory displays have the potential of greatly
enhancing presence.
Table 8.1 The 2D audiovisual reality–virtuality matrix. Only the cases in bold – REAL and
VIRTUAL – can be considered as non-MR situations