Dubois E., Gray P., Nigay L. (Eds.) The Engineering of Mixed Reality Systems
Подождите немного. Документ загружается.
Chapter 17
Enhancing Health-Care Services with Mixed
Reality Systems
Vladimir Stantchev
Abstract This work presents a development approach for mixed reality systems
in health care. Although health-care service costs account for 5–15% of GDP in
developed countries the sector has been remarkably resistant to the introduction of
technology-supported optimizations. Digitalization of data storing and processing in
the form of electronic patient records (EPR) and hospital information systems (HIS)
is a first necessary step. Contrary to typical business functions (e.g., accounting or
CRM) a health-care service is characterized by a knowledge intensive decision pro-
cess and usage of specialized devices ranging from stethoscopes to complex surgical
systems. Mixed reality systems can help fill the gap between highly patient-specific
health-care services that need a variety of technical resources on the one side and
the streamlined process flow that typical process supporting information systems
expect on the other side. To achieve this task, we present a development approach
that includes an evaluation of existing tasks and processes within the health-care
service and the information systems that currently support the service, as well as
identification of decision paths and actions that can benefit from mixed reality sys-
tems. The result is a mixed reality system that allows a clinician to monitor the
elements of the physical world and to blend them with virtual information provided
by the systems. He or she can also plan and schedule treatments and operations in
the digital world depending on status information from this mixed reality.
Keywords Mixed reality · Surgical sector · Location awareness
17.1 Health Care and Mixed Reality Systems
Physician and nursing staff in hospitals have a very high workload in general and
they have to handle many different problems in a very short time. Digital assistance
through software systems and electronic devices is able to reduce administrative
V. Sta n t chev (B)
Public Services and SOA Research Group, Berlin Institute of Technology and Fachhochschule
für Ökonomie und Management, Berlin, Germany
e-mail: vstantch@cs.tu-berlin.de
337
E. Dubois et al. (eds.), The Engineering of Mixed Reality Systems, Human-Computer
Interaction Series, DOI 10.1007/978-1-84882-733-2_17,
C
Springer-Verlag London Limited 2010
338 V. Sta n t chev
workload and free physicians and nurses for their core competence, taking care of
patients. Examples for such artifacts are hospital information systems (HIS) and
electronic patient records (EPR) or electronic health records (EHR) [4] which sim-
plify the access to patient data and medical information. Another artifact is devices
which provide information about the location of patients, staff, and medical devices
in the hospital. Such localization can be done via different technologies, for exam-
ple, Ultra Wide Band (UWB), Bluetooth (BT), or Wireless LAN (WLAN) location
applications.
Mixed reality systems can help fill the gap between highly patient-specific health-
care services that need a variety of technical resources on the one side and the
streamlined process flow that typical process supporting information systems expect
on the other side. Thereby, elements of the physical world in the clinic (e.g., patients,
devices, rooms, procedures and treatments) are combined with the digital world –
the IT and IS infrastructure, including processing and data elements. Therefore,
our development approach starts with an evaluation of existing tasks and processes
within the health-care service and the information systems that currently support
the service. Then we identify decision paths and actions that can benefit from mixed
reality systems. The result is a mixed reality system that allows a clinician to mon-
itor the elements of the physical world and to plan and schedule treatments and
operations in the digital world. This scheduling is then used in the physical world
to actually perform these actions, thereby integrating with computer-assisted med-
ical interventions (CAMI) systems. According to the OPAS/ASUR design notation
[18, 19], this system can be classified as an augmented reality (AR) system with
general assumptions as follows: (i) objects (components R) are patients, clinicians,
or medical devices; (ii) person (Component U) is the clinician in charge of planning
and scheduling treatments; (iii) adapters (Components A) are WLAN and RFID
tags; ( iv) system (Component S) is the HIS system of the clinic.
To address typical design requirements of such systems (e.g., cooperating ser-
vices, performance, and availability) we propose a service-oriented architecture
(SOA) as software architecture and architectural translucency to provide stable QoS.
Therefore, in the design and implementation phase of our approach, we evaluate
related technologies and QoS assurance approaches and then present design prin-
ciples for mixed reality systems. Furthermore, we present a clinical application
scenario where we applied our approach and developed a mixed reality system that
integrates the ASUR elements, implements the relationships between them, and
supports the services provided by the clinic. It provides a fusion of the physical
and digital worlds and thereby enables better handling of patients. Results demon-
strated increased satisfaction among patients and clinicians, as well as significant
cost s avings.
17.1.1 Augmented and Mixed Reality
Azuma [2] defines augmented reality (AR) as a variation of virtual reality (VR).
While VR technologies present to the user a completely synthetic environment
17 Enhancing Health-Care Services with Mixed Reality Systems 339
without a relation to the real world (the users perceives only the virtual world),
in an AR world the user can see the real world with its artifacts, together with
virtual objects superimposed upon or composited with the real world. Obviously,
AR supplements and enhances reality, rather than completely replacing it. Contrary
to authors who interpret AR primarily in the context of a head-mounted display
(HMD), Azuma defines three characteristics of AR systems:
• they combine real and virtual,
• they are interactive in real time, and
• they are registered in 3-D.
This definition of an AR system is extended by the definition of a mixed reality
system as an interactive system combining physical and digital entities [19]. There
is one important difference between these two definitions – while an AR system
deals primarily with the output (e.g., visual, 2-D, 3-D) of a system, a mixed reality
system deals also with inputs from the real world to the system (e.g., localization
and context awareness). Therefore, a mixed reality system generally differentiates
between input and output adapters. This is particularly true in our application sce-
nario where we focus on localizers as input adapters that monitor proximity and
exchange data with the system.
17.1.2 Usability Evaluation Techniques
“One of the basic lessons we have learned in human-computer interaction (HCI) is
that usability must be considered before prototyping takes place” [23]. This state-
ment is even more valid in life-critical applications such as health care. Nevertheless,
usability studies are still not considered an obligatory part of design in t his
domain.
A comprehensive overview of usability evaluation techniques is presented in
[23]. It differentiates between inspection methods (heuristic evaluation, cognitive
walkthrough, and action analysis) and test methods (thinking aloud, field obser-
vation, and questionnaires). These techniques are categorized according to their
applicability in different phases of the system development process, to their time
requirements and the number of users, evaluators, and the complexity of equip-
ment needed for the evaluation, as well as to their intrusiveness. A historic overview
and recent developments in usability research of augmented and mixed reality
specifically in the health-care domain is presented in [5].
Our approach considers intrusiveness as a particularly important aspect in health
care; therefore, we apply cognitive walkthrough and action analysis as inspection
methods. This requires high expertise from the evaluators, who are either clinicians
themselves or design specialists with extensive domain knowledge. As test methods
we use questionnaires and expert interviews.
340 V. Sta n t chev
17.1.3 Security Aspects
An overview of security aspects in ubiquitous computing in the health-care domain
is presented in [41]. Authors s tart with the strict legal requirements for patient-
related data and propose the usage of RFID tags for user authentication in mobile
and ubiquitous scenarios deployed at Graz University Hospital. Important identified
problems were security and privacy issues (protection precautions, confidentiality,
reliability, sociability). The authors also pointed to the need of further psychological
and technological research to address these problems.
Our approach follows these recommendations and incorporates security aspects
at the hearth of the mixed reality system. It employs user proximity as an addi-
tional level of security and patient data protection besides state-of-the-art role-based
security.
17.1.4 Work Structure
The rest of this chapter is structured as follows: in Section 17.2 we give an overview
of the development approach. Section 17.3 describes a description of the case study
and the scenario that underlies it. Furthermore, it defines critical aspects and design
requirements for mixed reality systems in health care and describes how these were
addressed within the case study. In Section 17.4 we assess our progress so far and
outline our future research activities.
17.2 Overview of the Development Approach
Our development approach (see Fig. 17.1) is iterative and covers the f ollowing
phases:
Fig. 17.1 Overview of the
development approach
17 Enhancing Health-Care Services with Mixed Reality Systems 341
1. evaluation of existing tasks and processes within the health-care service;
2. evaluation of the information systems that currently support the service;
3. identification of decision paths and actions that can benefit from mixed reality
systems; and
4. design and implementation of the mixed reality system, with focus on integration
and NFPs.
The process is iterative, so that we go through these phases when new require-
ments need to be reflected by the system. The following sections will describe each
of these phases in more detail.
17.2.1 Process Evaluation of the Health-Care Service
In order to capture the current state of the health-care service we use a standard
process-oriented approach – system analysis [24]. It consists of several steps as
depicted in Fig. 17.2 and is designed as a general blueprint for process optimization
projects [ 24].
For the process evaluation of the health-care service, we focus on the situation
analysis part of the approach. Here we use notations such as event-driven process
chains (EPCs) and business process modeling notation (BPMN). We use usability
evaluation techniques such as cognitive walkthrough, action analysis, field observa-
tion, and questionnaires [23]. Our user group were clinicians from hospitals with
large surgical departments, as well as specialized clinics (e.g., in minimally invasive
surgery).
Fig. 17.2 Process model of
system analysis
17.2.2 Evaluation of the Existing Information Systems
In this phase we focus on the evaluation of the information systems (IS) that cur-
rently support the service. This activity is the second main aspect of the situation
342 V. Sta n t chev
analysis [24]. During this phase, we enhance the process models from Section
17.2.1 with information about the information systems used at every step of the
process. This gives us a complete set of the current process activities, decision
paths, and information systems, currently in use for the provision of the health-care
service.
17.2.3 Identification of Decision Paths and Actions
That Can Benefit from Mixed Reality Systems
The starting point for this phase is the complete set of activities, decision paths,
and information systems that we compile in Section 17.2.2. We first classify them
to map the ASUR-design approach. We then discuss with clinicians who fall in the
user (Component U) category which decision paths and actions are currently under-
performing and would benefit from a mixed reality system. Here we also identify
relationships between the ASUR components (⇒ , → , =). The result is a list of
decision paths and actions that will be addressed by the mixed reality system in the
current iteration of the design and implementation process.
17.2.4 Implementation of the Mixed Reality System
We have already classified process attributes and information systems as ASUR
components in Section 17.2.3. Furthermore, we have a list of decision paths and
actions we want to address. In this phase our focus lies on the selection of proper
adapters and the overall integration in the IS environment of the clinic. We use
service orientation as an architectural model and the assurance of NFPs is a key
aspect in such distributed loosely coupled environments.
We elaborate in the case study (Section 17.3) the steps of these development
approach and how we apply them in the implementation of the system.
17.3 System Design and Implementation
Our application scenario focuses on the surgical sector – one of the largest cost
factors in health care and at the same time a place where high creation of value
takes place. For the effective utilization of the surgical sector pre- and postop-
erative processes are crucial. The surgery (equipments and specialists) is a fixed
(and expensive) resource. So pre- and postoperative processes need to be aligned
and provide for an optimized utilization of this resource. In this section, we show
exemplary how we generate scenario descriptions as text, then convert them to
the EPC notation, and use this EPC notation to create the ASUR model of the
system.
17 Enhancing Health-Care Services with Mixed Reality Systems 343
17.3.1 Design of the System
There are many visions how to redesign and reorganize perioperative patient flow
and work processes for maximum operating room productivity, which also bring
changes in operating room architecture [32].
Figure 17.3 shows an overview of perioperative and postoperative processes in
our application scenario. These were evaluated using our approach as described in
Section 17.2 and the usability evaluation techniques described in Section 17.1.
The perioperative processes start with a notification from an operating room
nurse or an anesthesia nurse that the clinical staff should transport the next patient
to the operating room. This action takes place in the ward (location area 1). Then a
transport service or a nurse moves the patient from the ward to the operating room
Fig. 17.3 Optimized process flow in surgery – the setting for the mixed reality system
344 V. Sta n t chev
area (location area 2). In the main registration area the clinicians transfer the patient
from the ward bed to an operating room table (location area 3). Next, the staff moves
the operating room table to the preparation room (location area 4), where the anes-
thesia nurse or the anesthetist prepares the patient for the operation. The stop t hat
follows is in the induction area, where the patient is anesthetized (location area 5).
Finally, the staff moves the patient to the operating room, where the preparation for
Fig. 17.4 A sample event-driven process chain within the application scenario
17 Enhancing Health-Care Services with Mixed Reality Systems 345
the operation starts, for example, operation-specific bedding, sterile coverage (loca-
tion area 6). After the operation ends the patient is taken out of the operating room
and back via the induction area (location area 5) and the preparation room (location
area 4) to the recovery room (location area 7). The process ends with the transport
of the patient back to the ward (location area 1).
The corresponding EPC model is shown in Fig. 17.4. It describes the sequence
of the actions, together with possible decision paths. The location areas for every
action are depicted with the same numbers as in Fig. 17.3.
An important prerequisite for a mixed reality system that supports this process is
object and person location sensing. Such localization can be done via different tech-
nologies, for example, Ultra Wide Band (UWB), Bluetooth (BT), or Wireless LAN
(WLAN) location applications. UWB has the advantage that it works independently
of other systems, allows very precise location, and transmits with a very low signal
strength. WLAN localization, on the other hand, can be done in existing WLAN
networks and allows the localization of computers, such as handhelds and laptops.
Hybrid approaches that use two or more such technologies typically provide higher
precision and more robust position sensing [8]. Handhelds and laptops are widely
used in hospital environments. They allow the access to hospital information sys-
tems and patient records. To improve the access to patient data the localization could
be combined with the data from the HIS and EPR. While doing the ward round the
laptop or handheld of the doctor could display only the patients of the room in which
the doctor currently is located. This would prevent the physician from searching for
the patient and therefore save time. Furthermore, such scenario would increase the
overall security and privacy of the system by providing an additional layer of secu-
rity to the role-based security model used in many existing health-care applications
– the access is allowed only for those members of the security group (e.g., doctors)
that are in physical proximity to the patient. We have described in detail the position
sensing techniques we use in [38, 34].
17.3.2 ASUR Model of the System
As already stated in Section 17.2 we use process models such as EPCs and BPMN
and then model their artifacts in the ASUR description [19]. The aim of our system
is to allow a person to assess the overall process by blending the real world of a
clinic with virtual information in a mixed reality system. Therefore, the adapters
need to allow the person to control actual movement of the real objects by mapping
it to the mixed reality system (see Fig. 17.5 for an overview and [34] for more details
about the location aspects).
17.3.2.1 Real Objects (Components R)
Objects can generally be patients, clinicians, or medical devices. Patients or clini-
cians typically trigger the events that start an event-driven process chain. Therefore,
they are mostly easily identifiable from the process models. Medical devices are