Dubois E., Gray P., Nigay L. (Eds.) The Engineering of Mixed Reality Systems
Подождите немного. Документ загружается.
x Contributors
Nadine Couture ESTIA-RECHERCHE, Technopole I zarbel, 64210 Bidart,
France; LaBRI, 351, cours de la Libération, 33405 Talence, France,
n.couture@estia.fr
Vincent Delfosse LUCID-ULg: Lab for User Cognition and Innovative Design –
University of Liège – Belgium, Vincent.Delfosse@ulg.ac.be
Emmanuel Dubois University of Toulouse – Tarbes, IRIT, 118 route de
Narbonne, 31062 Toulouse Cedex 9, France, Emmanuel.Dubois@irit.fr
Armin Duff SPECS@IUA: Laboratory for Synthetic Perceptive, Emotive, and
Cognitive Systems, Universitat Pompeu Fabra, Tànger 135, 08018 Barcelona,
Spain
Sophie Dupuy-Chessa Laboratory of Informatics of Grenoble, Grenoble
Université, 385 rue de l a bibliothèque, B.P. 53, 38041 Grenoble Cedex 9, France,
Sophie.Dupuy-Chessa@imag.fr
Florian Echtler Technische Universität München, München, Germany
Martin Flintham Mixed Reality Lab, Computer Science, University of
Nottingham, Jubilee Campus, Nottingham, NG8 1BB, UK, mdf@cs.nott.ac.uk
Areti Galani International Centre for Cultural and Heritage Studies, Newcastle
University, Bruce Building, Newcastle upon Tyne, NE1 7RU, UK,
areti.galani@ncl.ac.uk
Guillaume Gauffre University of Toulouse, IRIT, 118 route de Narbonne, 31062
Toulouse Cedex 9, France, Guillaume.Gauffre@irit.fr
Reinhard Gerndt University of Applied Sciences Braunschweig/Wolfenbuettel,
Wolfenbuettel, Germany, r.gernd@fh-wolfenbuettel.de
Martin R. Gibbs Interaction Design Group, Department of Information Systems,
The University of Melbourne, Parkville, VIC, Australia,
martin.gibbs@unimelb.edu.au
Guillaume Godet-Bar Laboratory of Informatics of Grenoble, Grenoble
Université, 385 rue de l a bibliothèque, B.P. 53, 38041 Grenoble Cedex 9, France,
Guillaume.Godet-Bar@imag.fr
T.C. Nicholas Graham School of Computing, Queen’s University, Kingston,
Canada K7L3N6, graham@cs.queensu.ca
Phil Gray University of Glasgow, DCS, Glasgow G12 8QQ, Scotland, UK,
pdg@dcs.gla.ac.uk
Timber Haaker Novay, 7500 AN Enschede, The Netherlands,
timber.haaker@telin.nl
Manuel Huber Technische Universität München, München, Germany,
huberma@in.tum.de
Contributors xi
Martin Inderbitzin SPECS@IUA: Laboratory for Synthetic Perceptive, Emotive,
and Cognitive Systems, Universitat Pompeu Fabra, Tànger 135, 08018 Barcelona,
Spain
Björn Johansson Department of Computer and Information Science, Linköping
University, SaabSecurity, Santa Anna IT Research Institute AB, Linköping,
Sweden, bjorn.j.e.johansson@saabgroup.com
Arne Jönsson Department of Computer and Information Science, Linköping
University, SaabSecurity, Santa Anna IT Research Institute AB, Linköping,
Sweden, arnjo@ida.liu.se
David Juras Laboratory of Informatics of Grenoble, Grenoble Université, 385 rue
de la bibliothèque, B.P. 53, 38041 Grenoble Cedex 9, France, David.Juras@imag.fr
Peter Keitler Technische Universität München, München, Germany,
keitler@in.tum.de
Eng Tat Khoo Mixed Reality Lab, 21 Lower Kent Ridge Rd, National University
of Singapore, Singapore 119077, Singapore, khooet@mixedrealitylab.org
Mendel Kleiner Department of Applied Acoustics, Chalmers University of
Technology, SE-412 96, Göteborg, Sweden, mendel.kleiner@chalmers.se
Gudrun Klinker Technische Universität München, München, Germany,
klinker@in.tum.de
Pontus Larsson Department of Applied Acoustics, Chalmers University of
Technology, SE-412 96, Göteborg, Sweden; Volvo Technology Corporation,
SE-405 08 Göteborg, Sweden, pontus.larsson@chalmers.se
Pierre Leclercq LUCID-ULg: Lab for User Cognition and Innovative Design –
University of Liège – Belgium, Pierre.Leclercq@ulg.ac.be
Sylvain Le Groux SPECS@IUA: Laboratory for Synthetic Perceptive, Emotive,
and Cognitive Systems, Universitat Pompeu Fabra, Tànger 135, 08018 Barcelona,
Spain
Jônatas Manzolli Interdisciplinary Nucleus for Sound Communications (NICS),
State University of Campinas, Rua da Reitoria, 165 – Cidade Universitária
"Zeferino Vaz" Campinas, São Paulo, Brasil
Zenon Mathews SPECS@IUA: Laboratory for Synthetic Perceptive, Emotive,
and Cognitive Systems, Universitat Pompeu Fabra, Tànger 135, 08018 Barcelona,
Spain
Tim Merritt Mixed Reality Lab, 21 Lower Kent Ridge Rd, National University of
Singapore, Singapore 119077, Singapore, tim@mixedrealitylab.org
Jan M.V. Misker V2_ Institute for the Unstable Media, Eendrachtsstraat 10, 3012
XL, Rotterdam, The Netherlands, jan@v2.nl
xii Contributors
Florian ‘Floyd’ Mueller Interaction Design Group, Department of Information
Systems, The University of Melbourne, Parkville, VIC, Australia,
floyd@floydmueller.com
Anna Mura SPECS@IUA: Laboratory for Synthetic Perceptive, Emotive, and
Cognitive Systems, Universitat Pompeu Fabra, Tànger 135, 08018 Barcelona,
Spain
Roderick Murray-Smith University of Glasgow, Glasgow, UK,
rod@dcs.gla.ac.uk
Laurence Nigay Grenoble Informatics Laboratory, 385 avenue de la Bibliothèque,
Domaine Universitaire, B.P. 53, 38 041 Grenoble cedex 9, France,
Laurence.Nigay@imag.fr
Susanna Nilsson Department of Computer and Information Science, Linköping
University, SaabSecurity, Santa Anna IT Research Institute AB, Linköping,
Sweden, susni@ida.liu.se
J.-L. Pérez-Medina Laboratory of Informatics of Grenoble, Grenoble Université,
385 rue de la bibliothèque, B.P. 53, 38041 Grenoble Cedex 9, France,
Jorge-Luis.Perez-Medina@imag.fr
W. Greg Phillips Department of Electrical and Computer Engineering, Royal
Military College of Canada, Kingston, Canada K7K7B4, greg.phillips@rmc.ca
Wayne Piekarski Wearable Computer Laboratory, University of South Australia,
Mawson Lakes, SA 5095, Australia, wayne@tinmith.net
Daniel Pustka Technische Universität München, München, Germany
Andrea Resmini University of Bologna, Via Carboni 2, 43100 Parma, Italy,
root@resmini.net
Patrick Reuter LaBRI, 351, cours de la Libération, 33405 Talence, France;
INRIA Bordeaux – Sud Ouest, 351, cours de la Libération, 33405 Talence France;
Université Bordeaux 2, 146 rue Léo Saignat, 33076 Bordeaux, France,
preuter@labri.fr
Dominique Rieu Laboratory of Informatics of Grenoble, Grenoble Université,
385 rue de la bibliothèque, B.P. 53, 38041 Grenoble Cedex 9, France,
Dominique.Rieu@imag.fr
Guillaume Rivière ESTIA-RECHERCHE, Technopole Izarbel, 64210 Bidart,
France; LaBRI, 351, cours de la Libération, 33405 Talence, France,
g.riviere@estia.fr
Luca Rosati University for Foreigners of Perugia, via XX settembre 25, 06124
Perugia, Italy, luca@lucarosati.it
Stéphane Safin LUCID-ULg: Lab for User Cognition and Innovative Design –
University of Liège – Liège, Belgium, Stephane.Safin@ulg.ac.be
Contributors xiii
Holger Schnädelbach Mixed Reality Lab, Computer Science, University of
Nottingham, Jubilee Campus, Nottingham, NG8 1BB, UK, hms@cs.nott.ac.uk
Rodrigo da Silva Guerra Graduate School of Engineering, Osaka University,
Osaka, Japan, rodrigo.guerra@ams.eng.osaka-u.ac.jp
J. David Smith School of Computing, Queen’s University, Kingston, Canada
K7L3N6, smith@cs.queensu.ca
Ross T. Smith Wearable Computer Laboratory, University of South Australia,
Mawson Lakes, SA 5095, Australia, ross@r-smith.net
Vladimir Stantchev Public Services and SOA Research Group, Berlin Institute of
Technology and Fachhochschule für Ökonomie und Management, Berlin,
Germany, vstantch@cs.tu-berlin.de
Jelle van der Ster V2_ Institute for the Unstable Media, Eendrachtsstraat 10,
3012 XL, Rotterdam, The Netherlands, jelle@v2.nl
Ellen Szwajcer Novay, 7500 AN Enschede, The Netherlands,
Ellen.Szwajcer@telin.nl
Ana Tajadura-Jiménez Department of Applied Acoustics, Chalmers University
of Technology, SE-412 96, Göteborg, Sweden, ana.tajadura@gmail.com
Bruce H. Thomas Wearable Computer Laboratory, University of South Australia,
Mawson Lakes, SA 5095, Australia, bruce.thomas@unisa.edu.au
Aleksander Väljamäe Department of Applied Acoustics, Chalmers University of
Technology, SE-412 96 Göteborg, Sweden; Research Laboratory for Synthetic
Perceptive, Emotive and Cognitive Systems (SPECS), Institute of Audiovisual
Studies, Universitat Pompeu Fabra, Barcelona, Spain,
aleksander.valjamae@iua.upf.edu
Daniel Västfjäll Department of Applied Acoustics, Chalmers University of
Technology, SE-412 96, Göteborg, Sweden; Department of Psychology, Göteborg
University, Göteborg, Sweden, daniel@ta.chalmers.se
Paul F.M.J. Verschure SPECS@IUA: Laboratory for Synthetic Perceptive,
Emotive, and Cognitive Systems, Universitat Pompeu Fabra, Tànger 135, 08018
Barcelona, Spain; ICREA, Technology Department, Universitat Pompeu Fabra,
Tànger 135, 08018 Barcelona, Spain, paul.verschure@iua.upf.edu
Frank Vetere Interaction Design Group, Department of Information Systems, The
University of Melbourne, Parkville, VIC, Australia, fv@unimelb.edu.au
John Williamson University of Glasgow, Glasgow, UK, jhw@dcs.gla.ac.uk
Christopher Wolfe School of Computing, Queen’s University, Kingston, Canada
K7L3N6, wolfe@cs.queensu.ca
Chapter 1
Introduction
Emmanuel Dubois, Phil Gray, and Laurence Nigay
1.1 Mixed Reality Systems: A Booming Domain
Human–Computer Interaction (HCI) is no longer restricted to interaction between
users and computers via keyboard and screen: Currently one of the most challenging
aspects of interactive systems is the integration of the physical and digital aspects of
interaction in a smooth and usable way. The design challenge of such mixed reality
(MR) systems lies in the fluid and harmonious fusion of the physical and digital
worlds. Examples of MR systems include tangible user interfaces, augmented real-
ity, augmented virtuality and embodied interfaces. The diversity of terms highlights
the ever growing interest in MR systems and the very dynamic and challenging
domain they define.
1.1.1 Variety of Mixed Reality Systems
The growing interest of designers and developers for mixed reality systems is due
to the dual need of users to both benefit from computers and stay in contact with the
physical world. A first attempt to satisfy this requirement consists of augmenting
the real world with computerized information: This is the rationale for augmented
reality (AR). Another approach consists of making the interaction with the computer
more realistic. Interaction with the computer is augmented by objects and actions
in the physical world. Examples involve input modalities based on real objects,
such as bricks also called tangible user interfaces and more generally augmented
virtuality (AV).
E. Dubois (B)
University of Toulouse, IRIT, 118 route de Narbonne, 31062 Toulouse Cedex 9, France
e-mail: emmanuel.dubois@irit.fr
1
E. Dubois et al. (eds.), The Engineering of Mixed Reality Systems, Human-Computer
Interaction Series, DOI 10.1007/978-1-84882-733-2_1,
C
Springer-Verlag London Limited 2010
2 E. Dubois e t al.
This anthology contains these two types of mixed reality systems. For exam-
ple, Chapter 6 considers tangible interaction, while Chapter 11 focuses on
augmented reality in the case of a mobile user.
1.1.2 Variety of Application Domains
There are many application domains of mixed reality, including game, art, architec-
ture and heath care. The variety of application domains makes it difficult to arrive at
a consensus definition of mixed reality systems, i.e. different people having distinct
goals are using the term “Mixed Reality”.
This anthology is an attempt to highlight this variety. The last part of the
anthology is dedicated to applications of mixed reality.
1.2 Mixed Reality Engineering
In this very dynamic research and industrial context, the goal of this anthology is to
go one step ahead to gain understanding of this challenging domain. The book cov-
ers topics on mixed reality engineering by addressing the design and development of
such systems: it includes chapters on conceptual modelling, design and evaluation
case studies, prototyping and development tools, and software technology.
The book brings together papers on a broad range of mixed reality engineer-
ing issues organized into three main sections:
• Interaction design
• Software design and implementation
• Applications of mixed reality.
1.2.1 Interaction Design
The first section dedicated to interaction design includes nine chapters.
1 Introduction 3
1.2.1.1 Elements of Design
• Chapter 2: An Integrating Framework for Mixed Systems. This chapter presents a
framework for comparing Mixed Reality Systems and exploring the design space.
It proposes the concept of mixed object as a unified point of view on MRS. Based
on intrinsic and extrinsic characteristics of mixed objects, this chapter illustrates
the use of this framework on several mixed systems.
• Chapter 3: A Holistic Approach to Design and Evaluation of Mixed Reality
Systems. This chapter focuses on the usefulness of a mixed reality system that
is complementary to the usability of the system. As part of an iterative user-
centred design approach, this chapter highlights the value of qualitative methods
in natural setting for evaluating the usefulness of a mixed reality system. Two
case studies of mixed reality systems and their qualitative experimental results
are presented.
• Chapter 4: Embedded Mixed Reality Environments. This chapter describes,
analyses and compares the context of the use of three different mixed reality
environments in terms of interaction space, asymmetries between physical and
digital worlds and social interaction triggered by these MRE before discussing
potential impacts of these dimensions on the interaction with such MRE.
• Chapter 5: The Semantic Environment: Heuristics for a Cross-Context Human–
Information Interaction Model. This chapter addresses the challenge of how to
support activities that involve complex coordination between digital and physical
domains, such as shopping experiences that combine both online and store-based
aspects. The authors present an information-oriented model that offers a frame-
work for the analysis of such mixed environments in terms of information and
product design, and spatial layout.
1.2.1.2 Specific Design Issues
• Chapter 6: Tangible Interaction in Mixed Reality Systems. This chapter discusses
the design of tangible interaction techniques, a particular form of mixed real-
ity environments. An engineering-oriented software/hardware co-design process
is introduced and illustrated with the development of three different tangible
user interfaces for real-world applications. Feedback collected from user studies
raised discussion about the use of such interaction techniques.
• Chapter 7: Designing a Mixed Reality Intergenerational Entertainment System.
This chapter focuses on the particular design issue of mixed reality for intergener-
ational family members (grandparents, parents and grandchildren simultaneously
using the mixed system). The focus is on intergenerational family entertain-
ment and the chapter presents a user-centred design process for designing a
collaborative Space Invaders mixed game.
• Chapter 8: Auditory-Induced Presence in Mixed Reality Environments and
Related Technology. This chapter explores how audio-based techniques can be
used to develop MR systems that enhance the sense of presence, one element of
comparison between the multiplicity and the heterogeneity of MR systems.
4 E. Dubois e t al.
• Chapter 9: An Exploration of Exertion in Mixed Reality Systems via the “Table
Tennis for Three” Game. This chapter explores dimensions relevant for the design
of MR systems that involve a user’s body in the interaction. A case s tudy is used
through the chapter to illustrate the different dimensions of the design of such
interaction.
1.2.1.3 Structuring the Design
• Chapter 10: Developing Mixed Interactive Systems: A Model-Based Process
for Generating and Managing Design Solutions. Using a classic model-based
transformational approach, the authors demonstrate via a worked example how
the ASUR family of models can be used to support t he progressive refinement of
a mixed reality system design from conceptual model to software realization.
1.2.2 Software Design and Implementation
The second section addresses technical platforms and solutions for interaction tech-
niques, development tools and a global view on the software development process.
This section i ncludes six chapters.
1.2.2.1 Technical Solutions and Interaction Techniques
• Chapter 11: Designing Outdoor Mixed Reality Hardware Systems. This chapter
presents the design of an outdoor mixed reality system from the point of view of
the involved hardware for localization, I/O interaction, computer systems and bat-
teries, and finally raises a set of lessons learnt related to technical considerations,
computing system management and I/O interactions.
• Chapter 12: Multimodal Excitatory Interfaces with Automatic Content Classific-
ation. This chapter presents a technical solution supporting the “active percep-
tion” concept: the goal is to encode information descriptor through the use of
vibrotactile and sonic feedback in response to user’s action on a device that
contains the information.
1.2.2.2 Platform: Prototyping, Development and Authoring Tools
• Chapter 13: Management of Tracking for Mixed and Augmented Reality Systems.
This chapter describes a platform, namely trackman built on top of the UbiTrack
middleware, that allows us to model a constellation of sensors for quickly defin-
ing a tracking setup based on a spatial relationship graph and a data flow diagram.
This is done in a graphical editor.
• Chapter 14: Authoring Immersive Mixed Reality Experiences. This chapter
presents a platform for authoring immersive mixed reality experiences for artistic
1 Introduction 5
use. The platform enables artists and designers to create the content for immer-
sive mixed reality environments. An example of a production, an art game done
by an artist along with a programmer, is presented.
• Chapter 15: Fiia: A Model-Based Approach to Engineering Collaborative
Augmented Reality. This chapter presents and illustrates a platform supporting
the implementation of a collaborative and distributed augmented reality system
on the basis of a high-level notation that models the system and abstracts the
distribution and software implementation considerations.
1.2.2.3 Life Cycle
• Chapter 16: A Software Engineering Method for the Design of Mixed Reality
Systems. Like the approach described in Chapter 10, the software engineering
process model presented here proposes that mixed reality systems are developed
via the progressive refinement of a set of linked models. Here the Symphony
development method has been augmented to accommodate mixed reality con-
cerns, including the physical aspects of interaction and the richer representational
options of mixed systems.
1.2.3 Application of Mixed Reality
The final section consists of a set of detailed case s tudies that highlight the appli-
cation of mixed reality in a number of fields, including emergency management,
games, and health care. The section includes five chapters.
• Chapter 17: Enhancing Health-Care Services with Mixed Reality Systems.This
chapter presents the steps of a development approach for enhancing health-care
services with mixed reality systems. The steps of the development approach are
illustrated by considering the design and development of a location-aware system
in the surgical sector of a hospital.
• Chapter 18: The eXperience Induction Machine: A New Paradigm for Mixed-
Reality Interaction Design and Psychological Experimentation. The eXperience
Induction Machine is a novel and rich mixed reality environment that offers a test
bed for immersive applications. The authors describe the environment, present a
justification for their system in terms of its role in exploring the psychological and
social aspects of mixed reality interaction and give two examples of experiments
performed using it.
• Chapter 19: MyCoach: In Situ User Evaluation of a Virtual and Physical Coach
for Running. The work presented in this chapter is rather different than others in
this anthology. MyCoach is a mobile system designed to work in concert with
a human coach in delivering assistance to amateur runners in improving their
performance. This study examines the potential of such an approach compared to
the non-augmented approach using a conventional human coach alone.