Dubois E., Gray P., Nigay L. (Eds.) The Engineering of Mixed Reality Systems
Подождите немного. Документ загружается.
346 V. Sta n t chev
Fig. 17.5 Overview of the ASUR components in the mixed reality system
mostly reflected as “devices” that are needed to complete an action within the pro-
cess chain. The completeness of the process model is therefore a key precondition
to identify them correctly within the ASUR design model.
17.3.2.2 Person as User (Component U)
A person is generally a clinician. He or she can be in charge of planning and
scheduling treatments, or more often this person is responsible for a function within
the specific process. A key aspect is that this person is normally not the one that
conducts the activities within the process chain where the information he needs
originates, but is rather the one that is in charge of other following functions within
the overall process.
17.3.2.3 Adapters (Components A)
The primary task of the adapters in our system is to map activities and move-
ments of the objects in the real world into the mixed reality system. Here we use
position-sensing techniques such as RFID and WLAN. These are input adapters,
more specifically localizers as specified in [19]. Figure 17.6 shows two examples.
The selection process for these localizers follows typical approaches for technology
selection and is described in detail in [34].
17 Enhancing Health-Care Services with Mixed Reality Systems 347
Fig. 17.6 Examples for localizers (courtesy of Ekahau)
17.3.2.4 Systems (Components S)
A typical system in our scenario is a HIS within the clinic which integrates with
ERP systems (taking care of resource planning) and CAMI systems.
The result is a mixed reality system that allows a clinician to monitor the elements
of the physical world and to blend them with virtual information provided by the
systems. He or she can also plan and schedule treatments and operations in the
digital world depending on status information from this mixed reality.
17.3.2.5 Relationships Between the ASUR Components of the System
A main objective of the mixed reality system is to use the existing infrastructure
(WLAN, RFID, HIS) to allow for clinicians to access patient and process data in a
context-aware way. These leads to several relationships between the ASUR compo-
nents of the system. Table 17.1 shows a selection. It shows examples of the three
kinds of relationships between ASUR components as defined in [19] – exchange of
data, physical activity triggering an action, and physical colocation. In this case, a
proximity of a patient object to a localizer adapter triggers exchange of data between
the adapter and the system.
Table 17.1 Sample ASUR relations of the mixed reality system
Textual description ASUR notation
Patient lies in bed (physical colocation) Robject
Patient
= Rtool
Bed
Patient closer than 1 m to localizer (physical activity triggering
an action)
Robject
Patient
⇒ A
in
Localizer transmits data to the system (exchange of data) A
in
→ S
17.3.3 Addressing Critical Aspects of Mixed Reality Systems
for Health-Care Services
Critical aspects of mixed reality systems for health-care services can be summed up
as follows:
348 V. Sta n t chev
• context awareness,
• timeliness and high assurance,
• fault tolerance, and
• interoperability.
This section describes how we addressed these aspects in the design and
implementation phase of our development approach.
17.3.3.1 Context Awareness
Only information that is relevant for the current specific task of the object (from the
ASUR taxonomy as specified in this section: patient, clinician, device) should be
provided.
This is an aspect that is often present in existing works, but is often not addressed
sufficiently.
One application that uses location-aware access to HIS is described in [31]. It
was implemented using agent-oriented middleware (SALSA) [39]. As the imple-
mentation is PDA based (device-based position calculation), neural networks were
used to implement position sensing. A key shortcoming is the insufficient precision
for delivering relevant patient information.
A concept of context-aware computing in health care is presented in [3]. It
includes a scenario with scenes such as Entering an Active Zone,theContext-aware
Hospital Bed, and the Context-aware Hospital EPR. Key lessons learned are the
following:
• Context awareness can improve targeted data access to EPRs by clinicians.
• Access to physical things (e.g., container, x-ray image, wheelchair, bed) reveals
activity. Based on information about this access, we can present proper data to
support t his activity.
• We can use context awareness to suggest courses of action, not to automatically
react to context changes.
17.3.3.2 Timeliness and High Assurance
The general term that comprise both timeliness and high assurance is availability.
There is no single agreed upon definition of this term. The historical view of avail-
ability has been as a binary metric that denotes whether a system i s “up” or “down”
at a single point of time. A well-known extension of this concept is to compute the
percentage of time, on average, that a system is available during a certain period –
this is how availability is defined when a system is described as having 99.99%
availability, for example.
There are several extended definitions of availability that address the inherent
limitations of this definition – availability should be considered as a spectrum,
rather as a binary metric, as systems can have various degraded, but still operational,
17 Enhancing Health-Care Services with Mixed Reality Systems 349
states between “up” and “down.” Furthermore, the definition does not consider QoS
aspects. One possible extended definition proposes to measure availability by exam-
ining variations in system QoS metrics over time [7]. In our implementation we
provide timeliness and high assurance using the concept of architectural translu-
cency [35]. We presented details about the implementation of the concept and an
evaluation in [36].
17.3.3.3 Fault Tolerance
Fault tolerance denotes the ability to provide a certain service even in the presence of
faults [26]. Fault-tolerance guarantees require assumptions about possible failures
and expected system load (load hypothesis). There exist a variety of fault models
at the hardware level of computer systems (e.g., gate l evel or functional level) [16,
14, 13] that cannot be applied directly to service fault tolerance in the context of
mixed reality systems, due to their higher complexity as compared to hardware-
implemented functions. Furthermore, there are fault models that are defined at the
component level of middleware environments such as CORBA [27, 21, 30]. This
work uses the fault model also used in [30]. The model was originally proposed in
[15] and extended in [25].
The fault model for our mixed reality system distinguishes the following fault
classes:
• Crash fault: A service loses its internal state or halts. The processor is silent
during the fault.
• Omission fault: A service fails to meet a deadline or to begin a task. This fault
class includes the crash fault.
• Timing of performance fault: A service serves a request before or after its
specified time frame of never. This fault class includes the omission fault.
• Incorrect computation fault: A service fails to produce a correct output in
response to a correct input. This fault class includes the timing fault.
• Authenticated Byzantine fault: A service behaves in an arbitrary or malicious
manner, but is unable to imperceptibly change an authenticated message. This
fault class includes the incorrect computation fault.
• Byzantine fault: Every possible fault. This class includes the authenticated
Byzantine fault.
The application and evaluation of this fault model is described in [33], its
implementation in health-care scenarios – in [36].
17.3.3.4 Interoperability
We use SOA as architectural paradigm. Interoperability within an SOA at the level
of the technical implementation is typically achieved by provision of service wrap-
pers and their subsequent composition to applications. Composition of applications
350 V. Sta n t chev
from web services is governed by different requirements than typical component-
based software development and integration of binary components. Application
developers and users do not have access to documentation, code, or binary com-
ponent. Instead, they rely only on a rudimentary functional description offered
by WSDL. Services execute in different contexts and containers, they are often
separated by firewalls and can be located practically everywhere.
With the native capabilities of web services fully developed, several approaches
for service composition started to emerge. The first-generation composition lan-
guages were Web Service Flow Language (WSFL) developed by IBM and Web
Services Choreography Interface (WSCI) developed by BEA Systems. However,
these proposals were not compatible with each other and this led to the develop-
ment of second-generation languages. The most popular of them is BPEL4WS [1],
which is a joint effort of IBM, Microsoft, SAP, Siebel, and BEA. It originates in
the combination of first-generation languages (WSFL and WSCI) with Microsoft
s
XLANG specification. Regardless of that, there is still no generally accepted stan-
dard for the process layer comprising aggregation, choreography, and composition
in the Web Service Architecture Stack (www.w3.org/2002/ws/).
We provide interoperability in two general ways:
• We use components that offer web service interfaces.
• We implement service wrappers for components that lack such interfaces.
For semantic interoperability we use concepts such as ontology mapping. A
detailed discussion of our implementation regarding semantic interoperability is out
of the scope of this work.
17.3.4 Addressing Software Design Requirements
Our implementation uses an SOA-based runtime infrastructure. The design princi-
ples presented in [3] for such kind of runtime infrastructure include
• distributed and cooperating services (e.g., an SOA),
• security and privacy,
• lookup and discovery (e.g., provided by foundation services in an SOA), and
• basic design principles such as performance and availability.
The rest of this section presents our design considerations that adhere to these
principles. These considerations constitute the transition between the first and the
second part of Phase 4 of our development approach (see Fig. 17.1).
17.3.4.1 Distributed and Cooperating Services
Services are autonomous, platform-independent entities. They can be described,
published, discovered, and loosely coupled in standard ways. Services reflect a
17 Enhancing Health-Care Services with Mixed Reality Systems 351
service-oriented approach to software engineering. It is a logical consequence of
the COTS (commercial off-the-shelf) component integration. It lets organizations
expose computing functionality programmatically over the Internet or various net-
works such as cable, the Universal Mobile Telecommunications System (UMTS),
xDSL, and Bluetooth using standard XML-based languages and protocols and a
self-describing interface [29].
Web services are currently the most promising service-oriented technology [40].
They use the Internet as the communication medium and open Internet-based stan-
dards, including the Simple Object Access Protocol (SOAP) [22] for transmitting
data, the Web Services Description Language (WSDL) [12, 11] for defining ser-
vices, and the Business Process Execution Language for Web Services (BPEL4WS)
[1] for orchestrating services. These characteristics are generally well suited for our
mixed reality system. Negative effects of the higher transmission overhead typi-
cally associated with the XML-based information interchange is compensated by
our application of the architectural translucency approach (see Section 17.3.3).
17.3.4.2 Security and Privacy
There are a variety of security mechanisms that can be incorporated in web service
architectures and that we considered for our mixed reality system. First, we can
secure the transmission using general approaches for securing web content such as
the Secure Socket Layer (SSL). We can use WS-Security [28] to secure SOAP mes-
sages and even extend this approach to complete BPEL compositions as proposed
in [10]. Authentication in such approaches is best provided by integrated solutions
for s ecurity credentials (e.g., Active Directory).
One successful approach for security in health-care applications is view-based
security [6]. It allows the definition of security policies using two central concepts
–aview (e.g., read diagnosis) and a role (e.g., a radiologist). The role gives the
mapping between a view and a system user (e.g., Dr. X).
With context awareness as a critical aspect, we can provide an additional layer of
security and privacy. This means that we introduce a location-based security model
on top of the view-based security model.
By combining these three security approaches (web services-based, view-based,
and location-based) we can guarantee appropriate security levels for different
applications of our mixed reality system.
17.3.4.3 Lookup and Discovery
Lookup and discovery are a central part of every SOA infrastructure, as already
described in this section. In our approach we use a COTS SOA suite such as the
WebMethods suite from Software AG.
1
It allows us to account for all major aspects
of service administration, lookup, and discovery, particularly when related with
1
http://www.softwareag.com/de/products/wm/default.asp
352 V. Sta n t chev
security and privacy aspects. This means that our security model is enforced even
before the search for dynamic service composition starts, thereby providing a list of
possible services that are functionally compatible, on the one side, and are allowed
for use by the specific role, on the other.
17.3.4.4 Performance and Availability
The use of an SOA suite and standard platforms for web services allows us
to dynamically provide additional resources to a web service, or start multiple
instances, if this is required at the current load of the service. Thereby, we can assure
at runtime the service levels that are specified together with the service definition in
the repository of the SOA suite. The approach is described in more detail in [37].
17.3.5 Technology Environment and Architectural Approach
Our WLAN positioning system is Ekahau [20], our HIS is FD Klinika [17], we are
using Tablet PCs as mobile devices for the clinicians. Our architectural approach
(see Fig. 17.7) is to use an SOA with wrappers that provide web service interfaces
to the enterprise service bus (ESB) [9]. The AT engine implements the concept of
architectural translucency in this scenario. It is responsible for service QoS assur-
ance by monitoring and management. When it notices that, for example, a service
is experiencing higher loads, it dynamically reconfigures the replication settings of
Fig. 17.7 Architectural approach
17 Enhancing Health-Care Services with Mixed Reality Systems 353
Fig. 17.8 Visualization of movement – tracking rails
the service to further provide the expected QoS. Integration of other systems (e.g.,
enterprise resource planning, external partner applications, health insurance sys-
tems). Representation and further information processing are depicted in the upper
part of the figure. During the first stage (depicted green) we plan to provide portal-
based access to EPRs that are extracted from the HIS and visualized on the Tablet
PC depending on the current location of the Tablet PC, particularly other WLAN-
enabled objects surrounding it (e.g., patient tags). During the second stage (depicted
red) we are introducing more complex planning and evaluation functions. These are
realized by composite services. Figure 17.8 shows an example – a clinician (e.g.,
an anesthetist or a surgeon) can follow the real movement of a patient and control
354 V. Sta n t chev
whether particular steps have been executed in the preceding functions of the pro-
cess flow. He or she can also access relevant patient data from these functions (e.g.,
when and how was the patient anesthetized, how were the vital parameters that were
last measured, and at which location have they been measured).
17.4 Conclusion and Outlook
This chapter presented a development approach for enhancing health-care services
with mixed reality systems. It integrates the ASUR design methodology to iden-
tify relevant components and their relations. The application of these concepts
was demonstrated using a case study in a clinical environment. The presented
development approach is iterative and covers the following steps: (i) evaluation of
existing tasks and processes within the health-care service, (ii) evaluation of the
information systems that currently support the service, (iii) identification of deci-
sion paths and actions that can benefit from mixed reality systems, as well as (iv)
design and implementation of the mixed reality system, with focus on integration
and NFPs.
We presented the ASUR components of the system and how we derive them from
process models specified in EPCs and BPMN. Furthermore, we evaluated critical
aspects of mixed reality systems that we had to address in our case study.
The case study described the application scenario, the architectural concept,
and the technologies we employed. It demonstrated the f easibility of implementing
mixed reality systems using the presented development approach.
Future work is focused primarily in the areas of definition of reference pro-
cesses for certain health-care services and corresponding design and implementation
guidelines.
Acknowledgments This work was developed in cooperation with several clinics where the field
tests have been conducted. There were valuable evaluation contributions from a range of experi-
enced clinicians. Finally, we would like to thank the anonymous reviewers and the editors of this
book for their constructive criticism and valuable improvement suggestions.
References
1. T. Andrews, F. Curbera, H. Dholakia, Y. Goland, J. Klein, F. Leymann, K. Liu, D. Roller, D.
Smith, S. Thatte, et al. Business Process Execution Language for Web Services (BPEL4WS)
1.1. Online: http://www-106. ibm. com/developerworks/webservices/library/ws-bpel, May,
139:140, 2003.
2. R.T. Azuma. A survey of augmented reality. Presence: Teleoperators and Virtual
Environments, 6(4):355–385, August 1997.
3. J.E. Bardram. Applications of context-aware computing in hospital work: examples and
design principles. In SAC ’04: Proceedings of the 2004 ACM Symposium on Applied
Computing, pages 1574–1579, New York, NY, USA, 2004. ACM.
4. R.J. Baron, E.L. Fabens, M. Schiffman, and E. Wolf. Electronic Health Records: Just around
the Corner? Or over t he Cliff? Ann. Intern. Med., 143(3):222–226, 2005.
17 Enhancing Health-Care Services with Mixed Reality Systems 355
5. R. Behringer, J. Christian, A. Holzinger, and S. Wilkinson. Some usability issues of aug-
mented and mixed reality for e-health applications in the medical domain. In HCI and
Usability for Medicine and Health Care, volume 4799 of Lecture Notes in Computer Science,
pages 255–266. Springer, 2007.
6. G. Brose. Manageable access control for corba. J. Comput. Secur., 10(4):301–337, 2002.
7. A. Brown and D.A. Patterson. Towards availability benchmarks: A case study of software
RAID systems. Proceedings of the 2000 USENIX Annual Technical Conference, 2000.
8. S. Bruning, J. Zapotoczky, Peter Ibach, and Vladimir Stantchev. Cooperative positioning with
magicmap. Positioning, Navigation and Communication, 2007. WPNC ’07. 4th Workshop on,
pages 17–22, March 2007.
9. D. Chappel. Enterprise Service Bus. O’Reilly, 2004.
10. A. Charfi and M. Mezini. Using aspects for security engineering of web service composi-
tions. In ICWS ’05: Proceedings of the IEEE International Conference on Web Services, pages
59–66, Washington, DC, USA, 2005. IEEE Computer Society.
11. R. Chinnici, M. Gudgin, J.J. Moreau, J. Schlimmer, and S. Weerawarana. Web Services
Description Language (WSDL) Version 2.0 Part 1: Core Language. W3C Working Draft, 26,
2004.
12. E. Christensen, F. Curbera, G. Meredith, and S. Weerawarana. Web Services Description
Language (WSDL) 1.1, 2001.
13. J.A. Clark and D.K. Pradhan. Fault injection. Computer, 28(6):47–56, 1995.
14. F. Corno, G. Cumani, M.S. Reorda, and G. Squillero. An rt-level fault model with high gate
level correlation. hldvt, 00:3, 2000.
15. F. Cristian, H. Aghili, R. Strong, and D. Dolev. Atomic broadcast: from simple message
diffusion to byzantine agreement. Inf. Comput., 118(1):158–179, 1995.
16. E.W. Czeck and D.P. Siewiorek. Effects of transient gate-level faults on program behavior.
Fault-Tolerant Computing, 1990. FTCS-20. Digest of Papers., 20th International Symposium,
pages 236–243, Jun 1990.
17. Fliegel Data. FD Klinika. http://www.fliegel-data.de/fd-online/, 2007.
18. E. Dubois, L. Nigay, J. Troccaz, O. Chavanon, and L. Carrat. Classification space
for augmented surgery, an augmented reality case study. Human-computer Interaction,
INTERACT’99: IFIP TC. 13 International Conference on Human-Computer Interaction, 30th
August-3rd September 1999, Edinburgh, UK, 1999.
19. E. Dubois, Philip D. Gray, and L. Nigay. Asur++: A design notation for mobile mixed
systems. In Mobile HCI ’02: Proceedings of the 4th International Symposium on Mobile
Human-Computer Interaction, pages 123–139, London, UK, 2002. Springer-Verlag.
20. Inc. Ekahau. Ekahau Positioning Engine 3.1. http://www.ekahau.com/file.php?id=129, 2005.
21. P. Felber and Priya. Experiences, strategies, and challenges in building fault-tolerant corba
systems. Transactions on Computers, 53(5):497–511, May 2004.
22. M. Gudgin, M. Hadley, N. Mendelsohn, J.J. Moreau, and H.F. Nielsen. SOAP Version 1.2.
W3C Working Draft, 2002.
23. A. Holzinger. Usability engineering methods for software developers. Commun. ACM,
48(1):71–74, 2005.
24. H. Krallmann, M. Schoenherr, and M. Trier. Systemanalyse im Unternehmen -
Prozessorientierte Methoden der Wirtschaftsinformatik. Oldenbourg, 2007.
25. L.A. Laranjeira, M. Malek, and R. Jenevein. Nest: a nested-predicate scheme for fault
tolerance. Transactions on Computers, 42(11):1303–1324, Nov 1993.
26. M. Malek. Responsive Computing. Kluwer Academic Publishers Boston, 1994.
27. E. Marsden, J.-C. Fabre, and Jean Arlat. Dependability of corba systems: Service characteri-
zation by fault injection. srds, 00:276, 2002.
28. A. Nadalin, C. Kaler, P. Hallam-Baker, R. Monzillo, et al. Web Services Security: SOAP
Message Security 1.0 (WS-Security 2004). OASIS Standard, 200401, 2004.
29. M.P. Papazoglou, P. Traverso, S. Dustdar, and F. Leymann. Service-oriented computing: State
of the a rt and research challenges. Computer, 40(11):38–45, Nov. 2007.