Dubois E., Gray P., Nigay L. (Eds.) The Engineering of Mixed Reality Systems
Подождите немного. Документ загружается.
76 H. Schnädelbach et al.
this might have on participants and nonparticipants alike. For instance, continuous
breaching of a rule or a norm might render them redundant or force an “organisa-
tion”, it being a museum, an office occupant or a street officer, to take action for
their r einforcement.
4.5 Conclusions
Our focus in this chapter has been the discussion of three example Mixed Reality
Environments and how specific design decisions enabled or hindered their tech-
nological, spatial, social and organisational embeddedness. The structure of the
discussion and the emphasis on selected topics and challenges emanates from the
specific MREs, which aimed to support informal interactions among friends, col-
leagues and strangers in the context of work and leisure. The comparison of the
three case studies suggested that the variation of degrees of embeddedness in the
physical environment, the activity at hand and the social and organisational context
may result in significantly different MREs. As ubiquitous, pervasive and MR tech-
nologies become deployed more widely and over longer periods of time, it will be
important to consider how they can meaningfully interact. This will have to con-
sider interoperability between those environments but more importantly the careful
consideration of how MREs can become embedded not just in physical settings but
also within the digital worlds that we increasingly make use of and their technical,
social and organisational norms.
Acknowledgments We thank the EPSRC for their support of the case s tudies through the
EQUATOR IRC [GR-N-15986]. This research is also supported by the Leverhulme Trust.
References
1. Azuma R T (1999) The Challenge of Making Augmented Reality Work Outdoors. In: Ohta Y,
Tamura H (eds.) Mixed Reality: Merging Real and Virtual Worlds. Springer, New York.
2. Benford S, Flintham M, Drozd A, Anastasi R, Rowland D, Tandavanitj N, Adams
M, Row-Farr J, Oldroyd A, Sutton J (2004) Uncle Roy All Around You: Implicating
the City in a Location-based Performance. In Proceedings of: ACE 2004, Singapore,
ACM Press
3. Billinghurst M, Baldis S, Miller E, Weghorst S (1997) Shared Space: Collaborative
Information Spaces. In Proceedings of: HCI International, San Francisco, USA, 7–10
4. Billinghurst M, Kato H (1999) Collaborative Mixed Reality. In Proceedings of: International
Symposium on Mixed Reality, Yokohama, Japan, 261–284
5. Billinghurst M, Kato H, Bee S, Bowskill J (1999) Asymmetries in collaborative wearable
interfaces. In Proceedings of: 3rd International Symposium on Wearable Computers, San
Francisco, USA, IEEE, 134–140
6. Bornik A, Beichel R, Kruijff E, Reitinger B, Schmalstieg D (2006) A Hybrid User Interface
for Manipulation of Volumetric Medical Data. In Proceedings of: IEEE Symposium on 3D
User Interfaces, 29–36
7. Bowers J, O’Brien J, Pycock J (1996) Practically Accomplishing Immersion: Cooperation in
and for Virtual Environments. In Proceedings of: CSCW, Boston, USA, ACM Press, 380–389
4 Embedded Mixed Reality Environments 77
8. Bowers J, Rodden T (1993) Exploding the Interface: Experiences of a CSCW Network. In
Proceedings of: Interchi, Amsterdam, The Netherlands, ACM Press, 255–262
9. Brown B, MacColl I, Chalmers M, Galani A, Randell C, Steed A (2003) Lessons From the
Lighthouse: Collaboration in a Shared Mixed Reality System. In Proceedings of: CHI Ft.
Lauderdale, USA, ACM Press, 577–584
10. Chalmers M, Galani A (2004) Seamful Interweaving: Heterogeneity in the Theory and Design
of Interactive Systems. In Proceedings of: DIS, Cambridge, USA, ACM Press, 243–252
11. Crabtree A, Rodden T (2008) Hybrid Ecologies: Understanding Cooperative Interaction in
Emerging Physical–Digital Environments. Personal and Ubiquitous Computing 12:481–493.
12. Drummond T, Julier S (2008) 7th IEEE and ACM International Symposium on Mixed and
Augmented Reality – ISMAR 2008. Vol. 2008. ACM Press, Cambridge, UK Conference web
site for ISMAR 2008
13. Falk J H, Dierking L D (1992) The Museum Experience. Whalesback Books, Washington,
DC.
14. Galani A, Chalmers M (2007) Blurring Boundaries for Museum Visitors. In: Marty P F, Jones
K B (eds.) Museum Informatics. Routledge, New York.
15. Greenhalgh C, Purbrick J, Snowdon D (2000) Inside MASSIVE-3: Flexible Support for Data
Consistency and World Structuring. In Proceedings of: CVE, San Francisco, USA, ACM
Press, 119–127
16. Harper R H R, Hughes J A (1993) “What a F-ing System! Send ‘em all to the same place
and then expect us to stop ‘em hitting”: Making technology work in air traffic control. In:
Button G (ed.) Technology in Working Order: Studies of Work, Interaction, and Technology.
Routledge, London
17. Heath C, Luff P (1992) Media Space and Communicative Asymmetries: Preliminary
Observations of Video-Mediated Interaction. Human Computer Interaction 7:315–346.
18. Koleva B, Schnädelbach H, Benford S, Greenhalgh C (2001) Experiencing a Presentation
through a Mixed Reality Boundary. In Proceedings of: Group, Boulder, USA, ACM Press,
71–80
19. MacColl I, Millard D, Randell C, Steed A (2002) Shared visiting in EQUATOR City. In
Proceedings of: CVE, Bonn, Germany, ACM Press, 88–94
20. Milgram P, Kishino F (1994) A Taxonomy of Mixed Reality Visual Displays. IEICE
Transactions on Information Systems E77:1321–1329.
21. Murtagh G M (2002) Seeing the “Rules”: Preliminary Observations of Action, Interaction and
Mobile Phone Use. In: Brown B, Green N, Harper R H R (eds.) Wireless World: Social and
Interactional Aspects of the Mobile Age. Springer, London
22. Nigay L, Coutrix C (2006) Mixed Reality: A Model of Mixed Interaction. In Proceedings of:
AVI, Venezia, Italy, ACM Press, 43–50
23. Randell C, Muller H (2001) Low Cost Indoor Positioning System. In Proceedings of:
UbiComp, Atlanta, USA, Springer, 42–48
24. Rodden T (2000–2007) EQUATOR Project. Vol. 2008, Nottingham, UK
25. Rogers Y (2006) Moving on from Weiser’s Vision of Calm Computing: Engaging UbiComp
Experiences. In Proceedings of: UbiComp, Orange County, USA, Springer, 404–421
26. Rudström Å, Höök K, Svensson M (2005) Social positioning: Designing the Seams between
Social, Physical and Digital Space. In Proceedings of: Online Communities and Social
Computing at HCII, Las Vegas, USA, Lawrence Erlbaum Associates,
27. Schmalstieg D, Fuhrmann A, Szalavari G H Z, Encarnacao L M, Gervautz M, Purgathofer
W (2002) The Studierstube Augmented Reality Project. Presence: Teleoperators and Virtual
Environments 11:33–45.
28. Schmalstieg D, Kaufmann H (2003) Mathematics And Geometry Education With
Collaborative Augmented Reality. Computers & Graphics 27:339–345
29. Schnädelbach H, Koleva B, Twidale M, Benford S (2004) The Iterative Design Process of
a Location-Aware Device for Group Use. In Proceedings of: UbiComp, Nottingham, UK,
Springer, 329–346
78 H. Schnädelbach et al.
30. Schnädelbach H, Penn A, Steadman P (2007) Mixed Reality Architecture: A Dynamic
Architectural Topology. In Proceedings of: Space Syntax Symposium, Istanbul, Turkey,
Technical University Istanbul, 10601–10614
31. Schnädelbach H, Penn A, Steadman P, Benford S, Koleva B, Rodden T (2006) Moving
Office: Inhabiting a Dynamic Building. In Proceedings of: CSCW, Banff, Canada, ACM Press,
313–322
32. Seichter H, Looser J, Billinghurst M (2008) ComposAR: An intuitive tool for authoring AR
applications. In Proceedings of: ISMAR, Cambridge, UK, IEEE, 177–178
33. Vogiazou Y, Raijmakers B, Geelhoed E, Reid J, Eisenstadt M (2007) Design for emer-
gence: experiments with a mixed reality urban playground game. Personal and Ubiquitous
Computing 11:45–58
34. Weiser M (1991) The Computer for the Twenty-First Century. Scientific American 265:
94–104.
Chapter 5
The Semantic Environment: Heuristics
for a Cross-Context Human–Information
Interaction Model
Andrea Resmini and Luca Rosati
Abstract This chapter introduces a multidisciplinary holistic approach for the gen-
eral design of successful bridge experiences as a cross-context human–information
interaction model. Nowadays it is common to interact through a number of different
domains in order to communicate successfully, complete a task, or elicit a desired
response: Users visit a reseller’s web site to find a specific item, book it, then drive
to the closest store to complete their purchase. As such, one of the crucial challenges
user experience design will face in the near future is how to structure and provide
bridge experiences seamlessly spanning multiple communication channels or media
formats for a specific purpose.
Keywords Bridge experience · Human–information interaction · Information
architecture · Information seeking · Findability · Internet of Things · Retail
design · Ubiquitous computing
5.1 Introduction
And indeed, for Aldrovandi and his contemporaries, it was all legenda – things to be read.
For Aldrovandi was meticulously contemplating a Nature that was, from top to bottom,
written. (Foucault [13])
In recent years phenomena like ubiquitous computing [56], the rise of an Internet
of Things [54], and issues related to information overload [57, 16] have clearly
demonstrated that a proper organization of information is becoming integral not
only to computer desktops, libraries, or web sites but to the physical environment as
well [33].
A. Resmini (B)
University of Bologna, Via Carboni 2, 43100 Parma, Italy
e-mail: root@resmini.net
79
E. Dubois et al. (eds.), The Engineering of Mixed Reality Systems, Human-Computer
Interaction Series, DOI 10.1007/978-1-84882-733-2_5,
C
Springer-Verlag London Limited 2010
80 A. Resmini and L. Rosati
Digital information is bleeding out of cyberspace and back into the real
world: Internet access moves into cell phones and hand-held devices; social
networks go mobile and constantly connect physically separated users; com-
mon daily objects become network enabled; shopping increasingly relies on web
sites, brick-and-mortar stores, and related user-generated content in the form of
reviews, comments, and personally selected links as common-use tools in a single
strategy.
As a consequence, human–information interaction across different environ-
ments is increasingly becoming a common day-to-day experience. Communicating
successfully; completing a task; eliciting a desired physical, mental, or emotional
response: All of these tasks often require users to move back and forth across
a variety of environments, interfaces, information architectures, and interaction
models.
Boundaries are blurring: Twitter is a micro-blogging application which lets user
broadcast 140-character-long messages to and from the Web using a number of
different devices. Is twittering friends from a cell phone about one’s dinner plans
interacting with the digital or the physical?
5.2 A Holistic Framework
Currently, we have no common cognitive frame of reference for the design of seam-
less interaction across different environments, and often across different experiences
inside the same environment. The Web and the store are treated like discreet blocks,
and this creates a heavy personal, cultural, and social burden [10]. User experience
design can relieve this by providing, structuring, and sustaining bridge experi-
ences: Coherent experiences capable of spanning multiple communication channels,
document genres, or media formats for a specific, tactical purpose [17].
This contribution outlines a holistic framework for the general design of success-
ful bridge experiences as a cross-context human–information interaction model for
the design of cognitive and informational continuity between digital and physical
spaces. We maintain this framework which improves user experience by allow-
ing users to rely on well-known, coherent behavioral, and cognitive patterns and
by encouraging their repeated application across environments and across different
layers of abstraction [23].
Bridge experiences rely on a synchronic model involving the user, space, and
time [50]. To this extent, the model identifies and describes four different ground
heuristics which act on the key aspects of the relationship. These are Resilience,
Place, Choice, and Correlation.
– Resilience is defined as the capability of a human–information interaction model
to shape and adapt itself to specific users, needs, targets, and seeking strategies.
– Place is defined as the capability to help users reduce disorientation and increase
legibility and way-finding in digital, physical, and mixed reality spaces.
5 The Semantic Environment 81
– Choice is defined as the capability to reduce the stress and frustration associ-
ated with choosing from an ever-growing set of information sources, services,
and goods.
– Correlation is defined as the capability to suggest relevant connections between
pieces of information, services, and goods to help users achieve explicit goals or
stimulate latent needs.
To illustrate how the model works, we will briefly introduce our view on the
current shift in design; we will describe in detail the four information architecture
principles and heuristics that constitute the framework; we will show how these
can be merged within a unified approach; and, finally, we will outline a prototype
design scenario for a semantically enhanced retail store, where different contexts
like web site, paper catalogs, and brick-and-mortar spaces serve different goals and
interfaces but share a common and unique information architecture and strategy to
provide seamless user experience and improved interaction.
Within the scope of this contribution, the terms context, domain, and environment
will be used interchangeably to describe the place where human–information inter-
action takes place; the term semantic layer or information layer will be used to refer
to all information in context, provided to users by means of digital technology, sig-
nage, or any combination of these. Similarly, the definition of semantic supermarket
is used for the information-enhanced shopping place where the presence of a seman-
tic information layer is made available to and exploitable by consumers. Although
this work is the result of a collaborative effort, Andrea Resmini wrote Sections 5.5,
5.6, 5.7, 5.8, and 5.9; Luca Rosati wrote Sections 5.1, 5.2, 5.3 and 5.4, and 5.5.2.
5.3 From Information Retrieval to Human–Information
Interaction
Information architecture (IA) is an emerging discipline and community of practice
focused on bringing principles of library science, design, and architecture to infor-
mation space [46]. Originally considered tightly connected to the World Wide Web,
it is now seen as a boundary field whose contributions are crucial where complex-
ity, unfamiliarity, and information overload stand in the way of the user [33, 34].
We consider this evolution of information architecture a strategic part of the general
shift from the design of objects to the design of processes [44, 50]. Contemporary
culture shows similar convergence patterns [20].
In the history of design, the two separated concepts of information retrieval and
interface are the product of a time when computers were limited, slow, and barely
able to interact with humans. They were significant when all interaction was in abso-
lute propositions and conveyed through a stand-alone computer screen: This is no
longer the case in a world where fast, connected micro-computers are everywhere.
Not surprisingly, since the late 1990s the focus has been shifting continually from
retrieval and interfaces to interactions and user experience [4, 10, 47].
82 A. Resmini and L. Rosati
While interfaces are bound to a specific single artifact, interaction models can
span a whole process involving a varying number of artifacts. A very good exam-
ple is the way Apple designed a common experience throughout the iPod/iTunes
line of products and are now actively bringing it to their stores [39]. Since the
process itself is a continuum, designing the interaction model globally allows to
build global, constant interaction patterns: Users are not forced to learn or relearn
diverging behaviors even if individual interfaces differ. This is why in the design
of bridge experiences interface design is not enough: Human–information interac-
tion issues have to be addressed throughout the whole process, incorporating the
apparent quirks of human behavior into information-seeking strategies [30].
This marks a shift from the traditional view of multichannel strategies [14] as
well, as the focus gets moved from optimized, data-driven information flows to
stable human–information interaction models [43].
Fig. 5.1 A physical–digital environment where our interactions with items and information are
tracked and saved to provide an improved user experience. A digital layer (middle) records the
interaction between the users (right) and the context (left). Image: A. Falcinelli
5.4 Resilience
I’ve often imagined gazes
surviving the act of seeing
as if they were poles,
measured distances, lances
in battle.
Then I think of a room
just abandoned
where similar traces remain
for a time, suspended and intersecting
in the balance of their design
intact and overlapping, like
pick-up-sticks. (Magrelli [28])
Our goals deeply influence our strategies when searching. Furthermore, cognitive,
cultural, and social models have a strong impact on behavior: As a result, differ-
5 The Semantic Environment 83
ent individuals browse and search differently because they follow different goals
and possess different models [3], and they might modify their patterns according to
context, time, and goals.
To work effectively then, an information space has to be able to adapt to chang-
ing information-seeking strategies [43]. Through the years, different techniques,
strategies, and technologies have been devised to solve or alleviate this issue in
specific domains and increase their findability: Faceted classification, ontologies,
the Semantic Web, folksonomies, and social classification. We think that these
approaches cannot be used as is in the cross-context design t hough, as they are
fundamentally asynchronous in a world becoming synchronic. A synchronic world
weaves stories through space and time, and every single noteworthy item generates
a story and a very large number of paths, which can be investigated, tracked, and
interpreted. These stories and paths do not form a dusty ink and paper archive, but
a vast data pool which can be manipulated in real time [50].
Being able to use it to its full extent is a fundamental opportunity and the real
challenge awaiting design: By observing and tracking the user’s paths and choices
inside the digital/physical environment, a system can adapt to their different navi-
gational models (that is, the physical layout of information, goods, and shelves can
be tuned and used to provide the correct pieces of information) even in real time,
and then leave these patterns available for social or personal reuse (Fig. 5.1). By
exploiting these data we can apply to physical spaces the hypertext logic and social
contexts which are currently peculiar to the Web (Fig. 5.2).
5.4.1 Scenario: The Resilient Supermarket
If Resilience in digital spaces is the capability of an information system such as
a web site to adapt and sustain different information-seeking strategies, when it is
applied to the physical context of the supermarket it is immediately apparent that
most of its outcome still remains information related and information based.
In this supermarket consumers use cell phones, hand-held devices, or a dedicated
reader to access the local information layer. This layer can comprise specialized
in-place information channels, as well as the official web site and any number of
independent sources. Through this, they can
– refind and retrace tasks and paths, including shopping tasks;
– personalize their tasks and paths;
– optimize their navigation, flow, and time spent shopping;
– create ad hoc paths around aisles, sales, and information; and
– receive suggestions and be informed of possible correlations among items.
The environment is now synchronic: It has not changed much in terms of general
layout or appearance, but it is aware of actions, space, and time.
84 A. Resmini and L. Rosati
Concretely, one of the possible strategies to acquire and build Resilience is to
capitalize on the flows and behavior of customers moving around, browsing, choos-
ing, and choosing not to choose by correlating these data. This is for example what
has been done with the EU SHAPE Living Exhibition Project [9]: Participants were
able to leave traces that were later used to potentially influence other people’s expe-
rience of the exhibit [8]. Similarly, by preserving the history of customer interactions
with the supermarket, a resilient layer can be offered to customers possessing an
appropriate device capable of both registering their shopping patterns and returning
them as structured, contextualized, social knowledge.
Resilience is not limited to be experienced through an augmented reality layer,
though. The supermarket can easily supplement this approach by empirically apply-
ing guidelines originally written for the design of web sites [45] to the physical
design of aisles and shelves to reinforce flexibility and adaptivity (Fig. 5.2). Sample
correspondences are briefly sketched out in Table 5.1.
Table 5.1 Web site–retail space correspondences map
Web site Supermarket
Main navigation Departments and aisles
Site map Interactive map of the supermarket
Alphabetic order Alphabetic display of products with their physical location
Shortcuts Preferred paths for returning users; discounted shopping;
healthy shopping; special regimes shopping
What is new Sales and promotions
Local or special navigation Theme paths: Pasta; Italian wines; Thai cuisine, etc.
In this scheme, every space and function is identified with a unique chromatic,
alphanumeric code:
– icons or geometrical shapes for departments;
– letters for aisles;
– numbers for racks or parts of racks, so that every product can be identified by its
letter + number combination code;
– colors for shortcuts and navigational themes, using either rack, shelf, ceiling, or
floor signage or any combination of these (Figs. 5.3 and 5.4).
Sales, promotions, and themed navigation are displayed in the main hall as well,
so that all options are immediately visible and available and users can simultane-
ously satisfy different cognitive models and personal search strategies. This favors
the passive gathering of information and products (push) as well, in accordance with
the principle of Least Effort [29] recalled by Bates [3].
1
1
Although the “Principle of Least Effort” was first discussed by Zipf in his seminal book Human
Behaviour and the Principle of Least Effort: An Introduction to Human Ecology, Mann was the
first to describe it in relationship with the information sciences.
5 The Semantic Environment 85
Fig. 5.2 Comparison map of interactions in a web site and in a supermarket
5.5 Place
“Here is Ts’ui Pên’s labyrinth,” he said, indicating a tall lacquered desk. [...]“Alabyrinth
of symbols,” he corrected. “An invisible labyrinth of time [...]” (Borges [5]).
The most important thing to know about Spimes is that they are precisely located in space
and time. They have histories. They are recorded, tracked, inventoried, and always associ-
ated with a story. Spimes have identities, they are protagonists of a documented process.
(Sterling [50]).
Space and place are different: Space is the base experience of our embodiment
[37], and it is objective, impersonal, and undifferentiated. Place, on the other hand,
involves a particular kind of presence that includes, besides physical space, mem-
ories, experiences, and behavioral patterns associated with the locale [8]: It is
personal, subjective, and communitarian. In a way, space exists independently of
man, while place cannot: And a sense of place is essential to our well-being as men
and women [51].
Although semantic space, digital space, or the Web do not map precisely or
exhaustively to a single physical representation of space [12], they can be places,
and they need to be places if they want to provide satisfying experiences. “People
goes where people is” [36]: As our interactions become often mediated by tech-
nology and devoid of physical presence, we tend to lose the ability to bond with
people, places, workplaces. As a side effect, the value of place has been reinforced