Reimagining situational awareness for decision makers by the implementation of MR technology and user-centered visual design.
My Role
The Team
Project Duration
Collaboration
Project Lead
Vision holder, technology experimentation and exploration, ecosystem design and ideation, research & ideation, prototyping, tech development, XR virtual production
Khushi Bhatt, Shailja Patel, Josh Gorski, Brittany Snider, Joel Sollomonson, Alex Hastings, Tianyi Zhang, Yiming Wang, Hanzhen Zhao, Wensi Huang, Lily Tang, Prof. SuAnne Fu, Prof. Jon Denham
January 2022 - March 2022
10 weeks
Deloitte
User Experience |
Immersive Reality |
Service Design
Mixed Reality
Situational Awareness
Microsoft Hololens 2
The Objective
We were asked to transform how the government envisions, develops, and sustains situational awareness and decision-making to establish Deloitte as the 21st-century firm of choice for Federal clients.
We were to develop a prototype that can demonstrate to senior decision-maker-level clients how to apply data to drive decision-making and risk-informed decisions, leveraging human-centered design. This prototype would have the ability to then productize the tool (on a platform that may contain a system of alerts/interfaces); sell on a subscription basis, with the delivery model to be later evolved to client needs.
We are to build this prototype based on the case study of COVID-19’s Omicron variant (2021-22) and its rapid assurance with endless data and a need for more prioritization.
Vertical Slices
Situational Awareness is visualized as the dashboard. Having an eagle’s eye view of the entire scenario is one of the important aspects of the process. Aided by visual cues on the most critical information helps the decision-maker to make decisions in a limited amount of time.
Second is Data Retrieval is where the decision-maker can pull in information from a huge pile of data that could be stored in the cloud, internal servers, or desktop itself. The features on the screen help the user make get this data with much ease.
And finally, the last area of focus is Risk Assessment, where the power of technology can be harnessed to help make more informed decisions. By testing the hypothesis in real-time, Mobius not only makes the most optimum decision but takes them with much confidence.
Our Vision
Through principles of visual perception, finely tuned spatialized audio, and explorations of immersive technology, this solution enables faster, more accurate decision-making within high-stress, time-pressed, crisis situations.
Our Solution
The Mobius system utilizes three main areas to deliver a situational system built off of core design elements. The three areas are:
Visual Design
To optimize visuals for quicker decision-making.
Technology Intervention
To enhance visualization and user experience through immersive technology.
Audio Design
To incorporate sounds to aid data comprehension and communication.
Mobius offers the framework for an innovative, multichannel situational awareness tool for tracking current data, as well as mitigation of risk through an analysis of projected data. Through research-backed visual guidelines, aural elements, and augmented and mixed realities, Mobius reimagines situational awareness.
Visual Design
The screens of the system were created to better illustrate hierarchy to help the decision-maker easily navigate across the device they are using to find the data essential in a limited amount of time. From this use of hierarchy through design elements and principles such as color, contrast, space, size, and so many more, Mobius reimagines data prioritization.
Technology Applications
The technology used in audio, as well as actual mixed reality devices, were used to reinforce our visual system. Each still allowing for our sense of hierarchy to apply. Our audio technology allows the hierarchy of information to be displayed in the actual ops center environment, and our mixed reality technology is used for a mobile use to communicate with others and have the ability to extrapolate pieces of data.
Mixed Reality via. Microsoft Hololens 2
Natural User Interface (NUI) enables data exploration, extraction, and manipulation in a mixed reality environment by visualizing three-dimensional dynamic information.
Dissecting the layers of information adds an element of embodied interaction, allowing the user to combine and customize the complexity of data viewing.
For the purposes of a proof of concept, we proposed 2 instances of MR in situational awareness.
1. SMART ESCALATIONS
A smart escalations feature for decision-makers reduces reaction times to critical situations. With in-built keyword extraction algorithm for Natural Language Processing (NLP), it auto-generates the key pieces of protocol and information required for a split second decision.
Muted for NDA purposes
Primary concept art
Final conceptual showcase
2. DIGITAL CARTOGRAPHY
A Geographic Information System (GIS) enhances the experience of exploring and analyzing geospatial data. This reimagines digital cartography for instant data access to all relevant and emerging information with real-time cause and effect simulations to enhance situational awareness.
Conceptual Prototype for Microsoft Hololens 2
HoloLens Mixed Reality prototype developed by Amir Ahmadi : https://www.asa989.org/
HoloLens Mixed Reality prototype developed by Amir Ahmadi : https://www.asa989.org/
Technology Justifications
Layers of information can be dissected and interacted with using NUI. User interfaces that allow for interactions using modalities such as touch, gestures or voice are often referred to as Natural User Interfaces.
This allows for embodied data exploration as the user can easily move the data and look at it from many different viewpoints to gain a richer, clearer understanding.
Natural User Interface - NUI
Telepresence
This allows for a collaboration with decision-makers or other relevant actors separated by distance. This information can be collectively shared and looked at organically by the user. By the use of telepresence, we eliminate the need for every single person to be looking at data from the same environment.
Z Axis Expansion
The introduction of the interactive Z-axis is the expansion of data visualization real estate. This allows for more layers of information and data relationships to be portrayed within one visualization. It also can help effectively communicate the cause and effect of relationships through a 360* infographic that can be more tangibly perceived with infinite zoom.
This is information that can be docked or anchored in physical space. This is a new way of organizing information by associating it with a space in the physical world to correspond with relevance, etc. It can help organize cognitive load by spatial prioritization.
The concept of Situated Analytics (SA) is the use of data representations organized in relation to relevant objects, places, and persons in the physical world for the purpose of understanding, sense-making, and decision-making.
Situated Analytics
