x SCADpro

My Role

The Team

Project Duration

Collaboration

Project Lead

Vision holder, technology experimentation and exploration, ecosystem design and ideation, research & ideation, prototyping, tech development, XR virtual production

Khushi Bhatt, Shailja Patel, Josh Gorski, Brittany Snider, Joel Sollomonson, Alex Hastings, Tianyi Zhang, Yiming Wang, Hanzhen Zhao, Wensi Huang, Lily Tang, Prof. SuAnne Fu, Prof. Jon Denham

January 2022 - March 2022

10 weeks

Deloitte

User Experience |

Immersive Reality |

Service Design

UX Case Study: Reimagining Situational Awareness with Mixed Reality

Client: Government sector via Deloitte

Project Name: MOBIUS

Overview: Reimagining situational awareness for decision makers by the implementation of MR technology and user-centered visual design.

The Objective

We were asked to transform how the government envisions, develops, and sustains situational awareness and decision-making to establish Deloitte as the 21st-century firm of choice for Federal clients.

In an era of endless data streams, decision-makers face challenges in prioritizing, understanding, and acting on critical information swiftly. Deloitte aimed to reimagine how governments manage situational awareness by leveraging Mixed Reality (MR) technology and user-centered design principles to create a next-generation solution.

Our goal was clear: transform crisis decision-making through an immersive prototype that showcases data-driven risk assessments and real-time decision support.

Objectives

🎯 Identify pain points in situational awareness for high-stakes decision-making.

🎯 Explore how MR technology can enhance understanding, communication, and action.

🎯 Validate a framework for a scalable, productizable prototype.

Methods

📝 Stakeholder Interviews: Conducted with senior decision-makers, analysts, and crisis response teams.

📝 Usability Studies: Explored current dashboards and decision-support tools in action.

📝 Competitive Benchmarking: Evaluated existing situational awareness tools.

📝 Secondary Research: Investigated COVID-19 data management challenges during the Omicron surge (2021-22).

Key Findings

✅ Cognitive Overload: Decision-makers are overwhelmed by non-prioritized data streams, delaying action.

✅ Fragmented Systems: Data sources are often siloed, making holistic insights difficult.

✅ Time-Critical Pressure: Visualizing potential outcomes in a compressed timeframe is a necessity.

✅ Need for Intuitive Design: Non-technical users prefer systems with a minimal learning curve.

Research PhasePrimary Research: Insights into Situational Awareness and Decision-Making Challenges

To inform the design of MOBIUS, we conducted extensive primary research, focusing on understanding the needs, frustrations, and workflows of our key stakeholders. Below is a breakdown of our methods and findings, including detailed data points, participant quotes, and affinity clusters to uncover deeper insights.

Research Methods

Stakeholder Interviews

• Participants:

  • 5 Senior Crisis Decision-Makers (e.g., government officials, executive-level managers).

  • 3 Data Analysts (federal and public health organizations).

  • 4 Field Agents (emergency response teams).

• Objective: Understand their workflows, frustrations, and technology gaps during crisis scenarios.

• Format: Semi-structured interviews lasting 45–60 minutes.

Usability Studies

• Participants: Same as above.

• Objective: Analyze current tools and systems used for situational awareness and decision-making.

• Approach:

  • Observed how participants interacted with dashboards and datasets.

  • Conducted task-based testing to evaluate speed, accuracy, and user satisfaction.

Surveys

• Participants: 30 government employees in crisis response roles.

• Objective: Gather quantitative insights into decision-making processes and technology needs.

• Focus Areas:

  • Frequency of data use in decision-making.

  • Time spent on data retrieval and prioritization.

  • Pain points with current tools.

Quantitative Data

📊 Decision Delays:

• 70% of decision-makers said data overload was their primary cause of delayed actions.

• Average time to identify critical insights: 20–25 minutes.

📊 Technology Challenges:

• 65% of analysts reported spending more time cleaning and organizing data than analyzing it.

• 50% of field agents cited difficulty accessing real-time data in remote locations.

📊 Collaboration Barriers:

• 80% of participants reported misalignment between centralized decision-makers and on-the-ground agents.

Key Findings from Primary Research

Qualitative Quotes

🗣️ Crisis Response Commander:

• “By the time I’ve found the right data, I’ve already lost precious minutes. It’s like looking for a needle in a haystack.”

• “I need a dashboard that thinks for me, surfacing what I need to know at a glance.”

🗣️ Data Analyst:

• “My job shouldn’t be 70% data cleanup and 30% analysis. Tools need to automate the grunt work so I can focus on strategy.”

• “Visualizing data for decision-makers is challenging because everyone wants something different, but we have limited tools to customize outputs.”

🗣️ Field Agent:

• “It’s hard to make fast decisions in the field when I’m working with outdated or incomplete data.”

• “Sometimes, I feel like I’m flying blind. Real-time updates would change the game.”

Affinity Clustering

After analyzing the interview transcripts and survey results, we grouped insights into affinity clusters to identify patterns. Below are the key clusters and associated themes:

Cluster 1: Data Overload

• Themes:

  • Difficulty prioritizing critical data.

  • Cognitive fatigue from interpreting raw datasets.

  • Over-reliance on manual filtering of information.

• Representative Quotes:

  • “I’m buried under too much data to act quickly.”

  • “It’s not just about having data—it’s about making sense of it fast.”

Cluster 2: Collaboration Gaps

• Themes:

  • Poor communication between centralized teams and field agents.

  • Limited shared tools for real-time collaboration.

  • Geographic separation creates misalignment.

• Representative Quotes:

  • “I can’t effectively share my analysis without sitting in the same room with decision-makers.”

  • “Our tools don’t allow for real-time syncing with the field.”

Cluster 3: Technology Shortcomings

• Themes:

  • Existing tools are slow and non-intuitive.

  • Lack of integration between systems (e.g., dashboards, GIS).

  • Inefficiency in retrieving historical and live data simultaneously.

• Representative Quotes:

  • “Switching between platforms wastes time we don’t have.”

  • “I wish there was one system that did everything.

Cluster 4: Visualization and Usability

• Themes:

  • Current dashboards lack visual hierarchy, making them harder to navigate.

  • Need for interactive, customizable visuals.

  • Desire for simplified interfaces with minimal learning curve.

• Representative Quotes:

  • “Dashboards should be intuitive—right now, they feel designed for engineers, not end-users.”

  • “A clean, organized interface makes all the difference in high-pressure situations.”

Actionable Insights from Research

⚡️ Critical Need for Prioritization:

• Design a system that surfaces the most critical data using visual and auditory cues.

⚡️ Real-Time Collaboration is Vital:

• Implement telepresence and shared interfaces for seamless decision-making across locations.

⚡️ Simplified Data Management:

• Automate data cleaning and organization to free up analyst time for higher-order tasks.

⚡️ Improved Usability through MR:

• Leverage spatial visualization (Z-axis, digital cartography) to reduce cognitive load and improve data comprehension.

User Personas

Situational Awareness Journey Map

Prototype: MOBIUS

The prototype incorporates three vertical slices:

1. Situational Awareness Dashboard: Displays high-priority alerts using spatialized visual and audio cues.

   • Situational Awareness is visualized as the dashboard. Having an eagle’s eye view of the entire scenario is one of the important aspects of the process. Aided by visual cues on the most critical information helps the decision-maker to make decisions in a limited amount of time.

2. Data Retrieval System: Enables intuitive exploration of massive datasets via Natural User Interfaces (NUI).

   • Second is Data Retrieval is where the decision-maker can pull in information from a huge pile of data that could be stored in the cloud, internal servers, or desktop itself. The features on the screen help the user make get this data with much ease.

3. Risk Assessment Tools: Includes SMART Escalations (NLP-based protocol generation) and Digital Cartography (GIS-based real-time mapping).

   • The last area of focus is Risk Assessment, where the power of technology can be harnessed to help make more informed decisions. By testing the hypothesis in real-time, Mobius not only makes the most optimum decision but takes them with much confidence.

Key Features

🗝️ Visual Hierarchy: Optimized layouts for faster comprehension of critical data.

🗝️ Mixed Reality: Interactive Z-axis visualizations using Microsoft HoloLens 2.

🗝️ Situated Analytics: Anchoring data in physical or virtual spaces to reduce cognitive load.

🗝️ Audio Design: To incorporate sounds to aid data comprehension and communication.

Visual Design

The screens of the system were created to better illustrate hierarchy to help the decision-maker easily navigate across the device they are using to find the data essential in a limited amount of time. From this use of hierarchy through design elements and principles such as color, contrast, space, size, and so many more, Mobius reimagines data prioritization.

User Testing

Eye-tracking

We conducted user tests with 5 participants without previous knowledge of situational awareness design. We tracked the participants’ eyes to where they looked and what order they were looking during the test. This document summarizes the user’s eye movement, showing the most common paths they took. The purpose of this test was to verify our designs followed our research around best practices for visual layout. We found that we were successful in our use of visual guidance, while not as successful in the case of the data retrieval page.

Technology Applications

The technology used in audio, as well as actual mixed reality devices, were used to reinforce our visual system. Each still allowing for our sense of hierarchy to apply. Our audio technology allows the hierarchy of information to be displayed in the actual ops center environment, and our mixed reality technology is used for a mobile use to communicate with others and have the ability to extrapolate pieces of data.

Mixed Reality via. Microsoft Hololens 2

Natural User Interface (NUI) enables data exploration, extraction, and manipulation in a mixed reality environment by visualizing three-dimensional dynamic information.

Dissecting the layers of information adds an element of embodied interaction, allowing the user to combine and customize the complexity of data viewing.

For the purposes of a proof of concept, we proposed 2 instances of MR in situational awareness.

Smart Escalations

A smart escalations feature for decision-makers reduces reaction times to critical situations. With in-built keyword extraction algorithm for Natural Language Processing (NLP), it auto-generates the key pieces of protocol and information required for a split second decision.

Muted for NDA purposes

Primary concept render

Final conceptual showcase

Digital Cartography

A Geographic Information System (GIS) enhances the experience of exploring and analyzing geospatial data. This reimagines digital cartography for instant data access to all relevant and emerging information with real-time cause and effect simulations to enhance situational awareness.

Conceptual Prototype for Microsoft Hololens 2

HoloLens Mixed Reality prototype developed by Amir Ahmadi : https://www.asa989.org/

Technology Justifications

Layers of information can be dissected and interacted with using NUI. User interfaces that allow for interactions using modalities such as touch, gestures or voice are often referred to as Natural User Interfaces.

This allows for embodied data exploration as the user can easily move the data and look at it from many different viewpoints to gain a richer, clearer understanding.

Natural User Interface - NUI

Telepresence

This allows for a collaboration with decision-makers or other relevant actors separated by distance. This information can be collectively shared and looked at organically by the user. By the use of telepresence, we eliminate the need for every single person to be looking at data from the same environment.

Z Axis Expansion

The introduction of the interactive Z-axis is the expansion of data visualization real estate. This allows for more layers of information and data relationships to be portrayed within one visualization. It also can help effectively communicate the cause and effect of relationships through a 360* infographic that can be more tangibly perceived with infinite zoom.

This is information that can be docked or anchored in physical space. This is a new way of organizing information by associating it with a space in the physical world to correspond with relevance, etc. It can help organize cognitive load by spatial prioritization.

The concept of Situated Analytics (SA) is the use of data representations organized in relation to relevant objects, places, and persons in the physical world for the purpose of understanding, sense-making, and decision-making.

Situated Analytics

Image Credits: Immersive Analytics, springwise.org

Image Credits: Immersive Analytics, springwise.org

Conclusion and Next Steps

The MOBIUS prototype demonstrated how MR and user-centered design can revolutionize situational awareness. It positioned Deloitte as a thought leader in cutting-edge crisis response technology, ready to scale the solution for Federal clients.

This case study highlights how MOBIUS embodies Deloitte’s vision of being the 21st-century firm of choice for Federal clients, seamlessly integrating innovation and user experience for meaningful impact.

From Research to Design

• Prioritization is Key: Visual and auditory cues significantly improved reaction times.

• Embodied Interaction: Users preferred interactive, 3D visualizations over flat dashboards.

• Collaborative Needs: Telepresence features improved decision-making in remote teams.

From Design to Testing

• Positive Feedback on NUI: Users found gesture-based data exploration intuitive.

• High Usability Scores: MOBIUS exceeded benchmarks in speed, accuracy, and user satisfaction.

Next Steps:

• Develop an MVP based on the prototype.

• Tailor features to specific government use cases.

• Evolve the delivery model for SaaS scalability.