The aim of
Auggmed

The aim of AUGGMED is to develop a serious game platform to enable single- and team-based training of end-users with different level of expertise from different organisations responding to terrorist and organised crime threats.

The platform will automatically generate non-linear scenarios tailored to suit the needs of individual trainees with learning outcomes that will improve the acquisition of emotional management, analytical thinking, problem solving and decision making skills. The game scenarios will include advanced simulations of operational environments, agents, telecommunications and threats, and will be delivered through virtual reality (VR) and mixed reality (MR) environments with multimodal interfaces.

Multimodal Interaction
(Three Modes)

The game scenarios will include advanced simulations of operational environments, agents, telecommunications and threats, and will be delivered through virtual reality (VR) and mixed reality (MR) environments with multimodal interfaces.

Trainers'
Tools

The game scenarios will include advanced simulations of operational environments, agents, telecommunications and threats, and will be delivered through virtual reality (VR) and mixed reality (MR) environments with multimodal interfaces.

Automated Game
Scenario Engine

At the heart of the AUGGMED platform is the Automated Game Scenario Engine (AGSE) defining the game logic for each training session based on the training objectives of individual trainees and teams. Trainers will be able to set up the learning objectives for individual responders and teams (e.g., police, counter-terrorist units, paramedics, etc.) and define an incident including location and type of threat. AGSE will create a terrorist-attack scenario taking into account the individual learning objectives and adapt scenario parameters in real-time to ensure that trainees engage in actions requiring demonstration of skills associated with the learning objectives.

The aim of Auggmed

AUGGMED aims to develop a serious game VR and MR platform that can be used for single and cooperative training in different settings offering different levels of realism and engagement. The platform will enable police, security forces and counter-terrorist units as well as first responders to train their staff in different VR environments with different scenarios and apply this training in the real infrastructure environment using mixed reality techniques.

In addition, the platform will be highly flexible offering a range of VR and augmented reality AR environments (from mobile devices to a full-scale real-life augmented environment) so that security forces can train at their own convenience as well as participating in well organised large-scale training sessions. This will deliver different learning experiences based on:

  1. The type of environment that the content is being accessed in.
  2. The capabilities of the devices themselves.
  3. The type of training being delivered.
  4. The number of people participating in the training session.
  5. The learning objectives of that particular session.

A multidisciplinary team with expertise in virtual/augmented reality, immersion, serious gaming, crowd and incident simulations, command and control, critical communications management and critical infrastructure protection, counter terrorism, emergency response, security and training will design and develop a new automated game scenario engine for training in scalable VR and MR environments involving single- or multi-players at home, office and/or on-site.

The platform will be designed to actively engage participants in interactive training sessions based on non-linear, adaptive scenarios supporting multiple outcomes. In addition, AUGGMED scenarios will be driven by effective simulations of CI environments, telecommunication capabilities, types of threat and agent behaviour (e.g., police, first responders, crowd and intruder) resulting in highly realistic consequences without real-life implications (safe failure). The platform will use the Unity3D engine to generate the VR environments and agents making it compatible with mainstream Sector tools and technologies.

The AUGGMED platform will be designed to address the learning objectives of the trainees including acquisition of problem solving, analytical and decision making skills. For that reason, the AUGGMED platform will incorporate tools that will enable trainers to set learning objectives, define their own scenarios, monitor training session, intervene in real-time to modify scenario and/or provide feedback, as well as evaluate trainees’ performance. Figure 1 shows the AUGGMED concept

Moreover, AUGGMED will include modes which will allow first responders to actively participate in training sessions using VR or MR environment with different levels of immersion and engagement. These modes will be suitable for both single- and cooperative team-based training. In addition, trainees can be in the same or different physical location.

  • Mode 1 - Basic Mode (Basic VR)
    Training anywhere using PC and/or portable devices with limited interactivity (e.g., no mobility or tactile feedback). Trainees can be in the same or different physical location. Trainees can interact with virtual and real agents (with avatar) who join the training session remotely.
  • Mode 2 - Intermediate Mode (Immersive Multimodal VR)
    •  Training anywhere using immersive VR (e.g., HMD with limited mobility and tactile feedback). Trainees can be in the same or different physical location. Trainees can interact with virtual agents and avatars of real agents who join the training session remotely.
    •  Training On-Site using projection based immersive VR (limited mobility and tactile feedback). Trainees can interact with virtual and real agents (with avatar) who join the training session remotely.
  • Mode 3 - Full Mode (Immersive Multimodal MR On-Site)
    Training On-Site using immersive MR (full mobility and advanced tactile feedback). Trainees can interact with simulated avatars as well as avatars controlled by other trainees who join the training session remotely.

The AUGGMED platform will be implemented to provide single- and team based training to prepare and defend against terrorist attacks on CI. CIs will be able to provide their security and training procedures and requirements as well as their CAD drawings and other mission critical information (e.g., vulnerability, risks and threats, security and first responders information, open space – underground and deep indoor critical communications predictions and analysis) onto the AUGGMED platform. Then, the AUGGMED platform will generate adaptive game scenarios for training and deliver them through different multi-modal interfaces and devices to security and first responder teams responsible for resolving crisis taken places at the specific CI. The CI will be able to deploy a single Mode or a combination of Modes to train the security personnel. For example, the operators of a transport terminal can schedule regular and frequent training drills using Mode 1 to keep a minimum the level of security staff readiness while scheduling Mode 2 and/or Mode 3 in irregular and less frequent intervals or whenever they feel that the staff involved are ready to proceed to the next level of training.

Furthermore, the AUGGMED platform will include tools that will enable the trainer to define scenarios, monitor a training session and provide feedback in real-time as well as evaluate the performance of individual trainees based on their learning objectives using appropriate metrics. AUGGMED will use Unity3D to generate the virtual environments which is a well-established and powerful environment offering interoperability with other serious game engines. AUGGMED will develop new simulations of Environments (CI), Threats (e.g., explosives, cyber-attacks, attacks using UAVs), and Agents (e.g., crowd, first responders, police units, intruders).