Simulating danger

Having moved far beyond entertainment and gaming, immersive technologies such as virtual reality, augmented reality, and mixed reality today cater to many industries, including education, healthcare, and manufacturing. Enterprises operating under challenging environments can adopt immersive extended reality (XR) technologies to simulate real-life hazardous scenarios for safety-related training. According to a report by Statista, the global XR market size is expected to reach $300 billion by 2024. Consequently, XR-based training plays a vital role across all industries.

Though XR has been around for decades, its adoption rate is gaining speed now, especially in manufacturing, aviation, mining, and oil and gas. Mining being one of the most hazardous environments, XR can play a vital role by enabling AR-based remote inspection and assistance, visually guided drilling operations, real-time heavy equipment operator assistance, assisted maintenance and repair of assets, and, above all, mine operator safety training.

XR can be leveraged for several kinds of safety training in the mining sector, including:

• Site familiarization
• Scenario-based rescue
• Historical incident-based rescue
• Emergency preparedness
• Equipment familiarization and operations
• Machine/Asset inspection and maintenance

In this paper, we focus on XR-based miner training, especially for India.

Leveraging AR-VR-MR for training

Immersive experiences can lead to better memory retention, according to the results of a 2018 study published in the journal Virtual Reality. GMetri reports a growth in knowledge retention of over 95% with spatial interactive learning, as against around 45% with conventional training videos.

Key elements of XR-based training can be developed using AR cards, VR-based training, and cave automatic virtual environment (CAVE). CAVE is an immersive high-resolution 3D stereoscopic projection, where images get directed at the six walls of a room-sized cube. It also includes solid-state illumination, motion tracking, interaction capabilities, and real-time 3D computer graphics. CAVE, in general, is suitable for multiple people to experience mining environments without individual VR headsets. 

Understanding the various types of immersive training possible with AR, VR, or MR is essential. Some of these are listed below.

Instruction manual/Guide: An immersive training module is usually built upon existing learning materials but with several options for interacting and capturing user inputs (simulations). For example, a 2D instruction manual of a machine part can be converted into a 3D visualization. It would allow the user to have a 360-degree view of the machine part with the instructions from the original manual. 

Full simulation: A full simulation would have the user navigate different scenarios, providing a holistic experience of the real-world asset/machinery. For example, a full simulation of the dashboard of a heavy earth moving machine (HEMM) could be developed for operators. This offers an opportunity for the user to get trained in complex machinery without stepping into it for real. It also allows the users to make mistakes and fail without risking the actual dangers of a real-life situation. 

Virtual explorer:  It allows users to explore an open cast or underground mine using a freeform approach. The user can freely navigate between numerous 3D models and interactive options, picking the most interesting ones. The virtual explorer can help new joiners or employees moving from one mine to another get acquainted with the terrain and specifics of the mine. Another example is AR/VR cards, where users could look at the recreation of a particular mining incident/accident to learn and understand the root causes and help avoid recurrence.

Remote training: Usually taken up in scenarios where the trainer or a senior expert trains a team member from a remote location. An example of this could be senior team members available at a different mining site/subsidiary, who teach someone from another site of the same organization. Another example could be HEMM and other machinery experts from original equipment manufacturers training first-level maintenance executives from the mining company.

Collaborative training: XR-based collaborative platforms can enable trainers and supervisors from different locations to collaborate and learn seamlessly in virtual environments with their photorealistic 3D avatars. Such platforms could be instrumental in scenarios where users from multiple mine locations or multiple subsidiaries (as in the case of Coal India Limited) want to run collaborative training through remote presence.

In addition to exposing workers to fewer real-world hazards during training, a key benefit of VR is greater efficiency. An XR-based training pedagogy provides a highly effective interim step between the classroom and the real world.

Below is a comparison of AR, VR, and MR for training in a mining environment and possible use cases.

AR/VR/MR used for training in a mining environment

Some illustrative use cases are captured in the table below, with the usage potential depicted as telecom towers. 

Proliferation of AR/VR/MR for training use cases in a mining environment

The illustrative use cases have been arranged in the chart below to show an overall ease of adoption. This is based on the complexity of development versus investment requirements. We can see that this ranges from the low-hanging fruits of past incident primers and technical support calls to the more complex CAVE creation.

Adoption of training use cases in a mining environment

Complexity of development includes factors such as technical difficulty, size of the project and project management efforts, skills availability, risks in development, and difficulty of user training/adoption.