Introduction

Virtual reality (VR) is a computer-generated scenario that simulates a realistic experience through which one interacts in a seemingly real or physical way (immersion) using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors. VR has mainly been based on interactive 3D graphics, user interfaces, and Visual Simulation (VS) (graphic representation of objects and systems of interest using graphical languages) to display relevant data and analyses on immersive spaces. VR, in general, is widely used in the fields of education and training due to its potential to stimulate interactivity and motivation [1]. Different studies about the use of virtual reality and immersive systems have evaluated the performance of this kind of system in a learning scenario [1], [2]. For vocational training aimed at adult workers, VR offers the possibility to move safely around dangerous places and for one to learn how to cope with emotions, while experimenting the best solutions while far away from real dangers [1]. The main motivation for VR use is that it gives the opportunity to learn in a realistic way via situations that are physically inaccessible due to time limitations, physical inaccessibility, danger, or ethical issues, among others.

Currently, the VR field is transitioning into work influenced by video games [1]. Much of the research and development conducted in the games’ community parallels the VR community’s efforts to focus on entertainment [3]. A video game is a mental contest, played with a computer, and follows certain rules for amusement, recreation, or winning. Video games have demonstrated a huge potential for entertainment; they can also generate abilities, skills, and knowledge by exploiting the main implicit characteristics that video games already have: attractiveness, affordability, and accessibility [4].

Serious games have become an active area to develop tools for training and education. As mentioned in [5], “A serious game is a digital game created with the intention to entertain and to achieve at least one additional goal (e.g., learning or health). These additional goals are named characterizing goals.” This kind of edutainment has been actively used in several areas and is described in Capdevila-Ibáñez, Marne and Labat’s work [6]. Some of the application fields include education on engineering ([7], [8], [9]), health ([10], [11], [12]), operation skills training ([6], [13], [14], [15]), and industrial plant operations ([16], [17]). Serious games are useful in a broad spectrum of application domains, ranging from educational games for younger audiences to collaborative training and simulation environments for the industry, as well as games for health and behavior change [1].

The simple presence of VR, immersion, and educational content in a game does not guarantee its effectiveness [2]. Rather, the educational power of any serious game also depends on a variety of other factors. More specifically, the educational content must be sound, age-appropriate, integrated effectively into the game, presented clearly, and the interface must be easy to use by the target audience. For this reason, it is necessary to follow an appropriate methodology to design a game that engages and accelerates the process of effective learning.

This paper presents how one of these state-of-the-art methodologies is used to develop serious games based on 3D models and immersive virtualization systems. Two vocational training areas are selected (Medical Imaging interpretation and laboratory machine operation) because training in these fields is time-consuming, requires access to delicate and expensive machines and tools that often are booked in normal operation, and sometimes it is not possible to have duplicate systems for training. The second section presents a state-of-the-art review focused on serious games development methodologies (conceptual and technical), as well as skill training applications. The third section briefly describes the CAVE system used on this project. The fourth section describes how one of the frameworks proposed on the literature is adapted to build our serious games. The system implementation (serious games and hardware integration) on the CAVE is presented in the fifth section. Finally, a discussion from a perceptual evaluation and the conclusions is presented in the sixth section.

State of the Art

Different methodologies have been proposed to generate effective serious games in a learning context [2]. [3], [4], [5], [6], [7], [8], [9]. Westera et al. [2] proposed a conceptual framework for a scenario-based game design based on three different levels of description (conceptual, technical, and practical levels). Yusoff et al. [3] provides another conceptual model for serious games based on learning theory focused on the design and measurement of achievement in meeting their learning outcomes. This process is dependent upon the tools used to develop the game; however, this methodology does not specify the different types of actors involved in the design process.

Van der Zee, Holkenborg and Robinson [4] provide a framework for the conceptual modeling of simulation-based serious gaming, which addresses relevant issues in a systematic, stepwise manner. In this framework, a methodology based on four steps is proposed. First, an ordered sequence of activities to undertake is declared. Then, the learning needs are stated, which include the definition of modeling objectives. Next, the game model inputs and outputs must be identified. Catalano, Luccini and Mortara [7] present a similar methodology that provides a set of recommendations that can be considered the assets through which the maximization of the learning impact can be pursued in addition to its effective evaluation.

Marfisi-Schottman, George and Frank [5] present a technical methodology focused on reusable software components. This seven-step framework shows the different actors and tasks to accomplish during the design process. These steps include: (1) specifying the pedagogical objectives; (2) choosing the serious game model; (3) describing the scenario; (4) searching for software components (reuse modules); (5) describing the scenario in detail; (6) ensuring pedagogical quality control; and (7) specifying subcontractors like a graphic designer, sound manager, actor, and others. Drummond, Hadchouel and Tesnière [8] propose a similar three-step framework for use during the development of a serious game about health. Usually, these frameworks, are not easily implemented because they focus on the conceptual level, leaving the technical and practical levels dependent of the final development. The methodology proposed by Capdevila-Ibáñez, Marne and Labat [6] is one of the clearer to follow in order to implement a new serious game. Capdevila-Ibáñez, Marne and Labat’s proposed methodology tries to bring closer conceptual ([2], [3], [4]) and technical [5] frameworks, and is composed of six facets: The specification of learning objectives, the domain simulation (game rules), the definition of interaction rules, the level design and difficulty progression, the decorum description, and the deployment of the game in an operational environment.

All these frameworks require a software capable of implementing them. Game engines [10] (a software that is extensible and can be used as the foundation for many different games without major modification) are the core software component that provide the underlying technology for creating serious games, simplifying development, and incorporating all of the elements vital in a game such as physics, collision detection, graphical user interface (GUI), artificial intelligence, network functionality, sound, and event engine [11], [12]. A good review of different game engines suitable for SG is presented in [13].

The GameObject System is a central element of most modern game engines that defines and manages the game objects. It also controls the messaging between them. Game objects represent any dynamic element within a game world [11] and can represent an artifact, actor, or concept like trees, doors, characters, etc. Game mechanics, however, denote the way the players can interact with the game and focus on the internal management of interactions. Gameplay denotes the external appearance of interactions [10] and is the context of operation defined by the rules of play and the physical and temporal boundaries of the game [2]. Rules are regulations and settings constraining the game, which typically take the form of if-then relations. Finally, the game logic is the collection of triggers and the underlying rules acting on the game world and its users.

Döner et al. [10] present the technical issues for the creation of a serious game (design, authoring, processes, tools, and content production). They also give some recommendations for development of serious games focused on two main components: Game design (conceptual) and game production (technical). Game production involves the selection or hardware and software tools to develop the game.

There are several examples of the use of game engines to develop serious games for different fields. These platforms facilitate the creation of game worlds and game objects by integrating tools to drive files from other software fields such as engineering, architecture, and medicine. Objects and worlds generated using conventional engineering Computer-Aided Design (CAD)/Computer-Aided Manufacturing (CAM) methodologies can be used to directly create custom serious games. In a similar fashion, 3D representations based on Medical Imaging (MI) can be directly incorporated into the game object creation workflow.

Barreto et al.’s work [14] outlines a methodology for creating virtual worlds with high adjustment to reality using 3D geometric models of high precision. This work incorporates BIM (Building Information Modeling) methodologies, which is a process for the generation and management of data of a building or infrastructure using real time three-dimensional dynamic modeling software. Other similar approaches have been presented in [17], [18], [19], [20]. In Hilfert, Teizer and König’s work [17], for example, it is presented how a BIM model was used to create the virtual environment using a game engine and create a mission based on rules and scoring for first-person point of view games. Kumar et al. [18] introduce the concept of a BIM game prototype that integrates BIM and gaming into architectural visualization. In general, these works detail different methodologies and programming flows to convert a BIM-type construction model into a model compatible with Virtual Reality Modeling Language (VRML).

As described in [21], there are three main focus areas to succeed in serious games: attractiveness, accessibility, and affordability. Wortley also presents a portfolio of technologies that includes interfaces, portable devices, sensors, 3D graphics, and artificial intelligence that are essential for the three areas.

The immersive virtualization technologies contribute to game attractiveness, allowing the user to experience the sensation of being physically present in a virtual three-dimensional environment [22], [23]. Different systems to realize this type of immersive environment have been used in education and research in recent years [22]. One such approach utilizes HDMs (head-mounted displays), in which a helmet projects independent views for each eye and images of the same scene are projected with a brief separation of the point of view, allowing the sensation of stereoscopy in the projected 3D world. With this system, the user experiences total immersion in the virtual world [10]. However, this system has the disadvantage that the user cannot interact directly with real objects or people (only virtual objects). Another drawback is that it can cause disorientation and visual fatigue with prolonged use.

On the other hand, there are immersive virtualization systems called CAVE (Cave Automatic Virtual Environment), in which there is a physical projection space usually in the form of a cube or cylinder where various projectors with stereo capability project images onto walls and the floor, surrounding several users that are inside the structure. The users wear active type stereo glasses synchronized with the projectors, allowing the user to perceive stereo images within the three-dimensional scene [22], [23]. In this type of space, the user can interact with the people and objects that are inside the projection space, since the glasses do not hinder vision and are only polarized to the images projected on the surfaces. Therefore, virtual or augmented reality applications are allowed. In this system, users usually also carry a signaling device, a joystick, or haptic interface combined with a motion capture system that allows the user to control the scene and interact with the simulation.

Figure 1 summarizes how these frameworks and technical methodologies can be fused to generate serious games. The main contribution of this paper is to show how to integrate a state-of-the-art serious game design framework with immersive technologies in order to create tools for vocational training aimed at adult workers, areas in which traditional training is expensive and time-consuming.

System

For this project a four-sided (“S4”) CAVE is used. Like most CAVEs, it provides viewers with an immersive experience by viewing and interacting with 3D virtual reality worlds, stereoscopic big-data displays, and 3D models of real-world locations and objects. Some of the facility’s features include:

• A 3.2 m wide by 3.2 m deep by 2 m high enclosure in which viewers can interact with a virtual world

• High-performance DLP projectors with active 3D stereo

• Active 3D glasses with DLP-Link synchronization for DLP 3D-ready projectors

• Up to 20 viewers can use the CAVE at one time

• Motion Capture MC system

• Haptic device interface (under implementation)

• Powered by a single Windows and Linux machine with a fast Intel Xeon CPU and an NVIDIA Quadro P5000 graphics engine

Figure 2 shows the CAVE used for this project at PUJ.

The CAVE generates immersion by using stereographic projection synchronized with polarized glasses. Users achieve the sensation of immersion because they are surrounded by perpendicular surfaces projecting stereo images, covering the field of view. System interaction is performed with a Microsoft® Kinect system, which identifies user position and gesture commands by means of skeleton and joint identification.

Hardware and software integration are depicted on figure 3. The core of the system is a WorkStation WS, which runs the Game Engine as well as the FAAST toolkit. As mentioned in next section, the FAAST middleware allows capturing user movements and interaction with the game engine. The game engine is configured to generate 3D images that are projected on the CAVE walls using two DLP projectors. These projectors are configured using a mosaic display option in the GPU settings, which allows the system to work as a single display. A 3D signal synchronizes active glasses operation using a 3D-Din VESA Cable. Therefore, the WS requires a GPU card with VESA 3D-Din socket as shown on figure 3. Active glasses are be paired with DLP projectors in order to synchronize left and right eye signal, enabling the 3D perception. User felt full immersion when the field of view is filled and the 3D signal is running properly.

Serious Game Development

Two vocational training areas have been selected: Medical imaging interpretation and laboratory machine operation, because training in these fields is time consuming, requiring access to delicate and expensive machines and tools that often are booked for normal workload. Thus, training based on serious games is a promising alternative in these fields. Additionally, for these specialties there are detailed 3D models of the application areas, which enable the natural generation of educational content suitable for these kind of educational tools.

As mentioned in state-of-the-art section, there are different methodologies that can be used to develop a serious game. Some of them are more conceptual and not easily implemented on a serious game engine. In the work presented by Capdevila-Ibáñez, Marne and Labat [6], they proposed a simple methodology suitable for our serious games composed of six facets: (1) specification of learning objectives, (2) definition of interaction or game rules, (3) level design, (4) difficulty progression, (5) decorum description, and (6) deployment of the game in an operational environment. This methodology is similar to the one proposed by Marfisi-Schottman, George and Frank [5] and is quite similar to the one used commonly on entertainment video games [10], [11], [12]. Our serious games have been developed using a methodology based on five stages: (1) learning objectives, (2) model description, (3) user interactions with GameObject, (4) problems and progression, and (6) decoration. This methodology is derived from the one presented in [6] with the difference that we considered the deployment as already included in the five stages.

Learning Objectives

This stage focuses on clearly defining what learners should have learned after playing the game. It is directly dependent on the specific application and field. Usually, it includes the skills and attitudes—and the relation between them, which can be represented using ontology or typed graphs. In our case, the learning objectives have been defined with a teacher (pedagogical expert [5]) to formalize the knowledge and behavior relevant to the game domain. The learning objective for the training game aims to generate skills for the safe operation of the lab machine according to the operations manual. The user must demonstrate the capability to turn on and operate the lab machines, considering safety issues and following the steps for correct operation.

In the case of medical games, the learning objective is for the student to be able to understand the different anatomical planes (axial, sagittal, and coronal orientations) that medical imaging has and how they correspond to different slices of the real bone structure of the body. In this kind of imaging, the sagittal or median plane, divides the body into left and right hemispheres. The coronal or frontal plane (vertical) divides the body into dorsal and ventral (back and front or posterior and anterior) portions. Finally, the transverse plane or axial plane (lateral, horizontal) divides the body into cranial and caudal (head and tail) portions. Understanding these planes and their compositions is fundamental for a radiologist to be able to read and understand medical images.

Model Description and Game Object

This stage uses external information to build a 3D model suitable for use on the game engine. This process is dependent on the type of application and uses specialized tools to create content [10]. However, the methodology explained below is replicable in several fields that can generate 3D models to create suitable game objects. In the case of infrastructure and facilities training, a BIM model is employed. There are several strategies to use and integrate BIM models into game engines. In this project, the BIM model is processed and exported using a methodology like the one presented in [18]. In this methodology, the BIM model is preprocessed and exported to a 3D model file suitable for import into the game engine (figure 4 shows an example of a BIM model of the CTAI lab at the first level). Penn University has developed a workflow that illustrates an effective way to link BIM models on Unity [24]. The current project used this methodology. First, the BIM model is exported (from .RFA REVIT®) to .FBX® file. This file for data exchange technology is a 3D asset exchange format that facilitates higher-fidelity data exchange between animation and games software. This file is preprocessed using 3DSMax® from Autodesk®. Open source software, such as Blender, can also be used at this stage [25] to preserve the properties of textures and materials. Next, the file can be directly imported into Unity as a new 3D object asset.

For Medical Imaging (MI) modeling, there are several software programs that allow processing this kind of dataset (DICOM file) to obtain a model file suitable for 3D printing (i.e., OBJ files). The Digital Imaging and Communication in Medicine (DICOM) Standard is used globally to store, exchange, and transmit medical images. The DICOM Standard incorporates protocols for imaging techniques such as radiography, computed tomography (CT), magnetic resonance imaging (MRI), ultrasonography, and radiation therapy. The open OBJ file format is a simple data format that represents 3D geometry as a list of vertices and texture vertices containing scale information in a human-readable comment line. It is also possible to import this OBJ file directly on several game engines as a new object asset. To have a suitable OBJ file, the medical images must be preprocessed and segmented to obtain different layers that correspond to internal structures of the body (blood vessels, muscles, bones, cartilage, tendons, ligaments, etc.). All of these elements can be assigned to specific layers, allowing them to have a composition for the final model. For this project, the Open Source medical software InVesalius® [26] is used to generate virtual reconstructions of body structures based on Computed Tomography CT or Magnetic Resonance Imaging (MRI). The first step of the process was to obtain the medical data (DICOM) through computed tomography; then, the images were processed to create a 3D model of the region of interest by the imaging processing software. Different filters and thresholds must be applied to different isolated body organs and tissues. This image processing usually depends on the expertise of the radiology technologist to obtain accurate models suitable for export to OBJ files. Figure 4 shows an example of a 3D cranium model generated using DICOM images. This file can be directly imported into Unity (Game Engine) as a new 3D object asset.

Once the .FBX file (BIM) or the .OBJ (medical imaging model) has been imported as a new asset on the game engine (figure 5), it can be edited in order to define animations and interaction properties on the serious game world. The animations are dependent upon the user’s interaction with the 3D model. Serious games are designed following specific storylines that establish interaction in activities and behavior rules (virtual world and user avatar) [12].

Storyboards are sequences of sketches or screen layouts that focus on the main actions and interactions in a possible situation. Storyboards take textual descriptions of task flows (such as scenarios) and turn them into visual illustrations of interactions.

User Interactions

User interaction with the serious game is performed using a Microsoft® Kinect camera to track the user position and gestures in the virtual scene. It is complemented with a conventional wireless gamepad for specific user actions. For this project, the Flexible Action and Articulated Skeleton Toolkit FAAST [27] was used. FAAST is a middleware to facilitate the integration of full-body control with games and VR applications. The toolkit relies upon software from OpenNI® and PrimeSense® to track the user’s motion using the PrimeSensor or the Microsoft® Kinect sensor. FAAST includes a custom VRPN server [28] to stream the user’s skeleton over a network, allowing VR applications to read the skeletal joints as trackers using any VRPN client. FAAST is free to use and distribute for research and non-commercial purposes. Custom gestures combining actions of specifics joints are mapped as a series of input events. These input events are the interactions that users can perform with the virtual environment (game management). Interaction includes gestures for displacement, rotation, jumping, taking, button triggering, among others. Gesture combinations include multiple relative joint positions or displacements associated with the user’s skeleton representation, as shown in figure 6.

Problems and Progression

Problems and progression is an iterative phase that defines the methodology used to increase the complexity of the game (levels). Decisions are based on a methodology like the one proposed in [6]. Decision trees or logic rules are defined for game evolution. Game states are used to evolve in time. At each game state, job systems or activities are defined. Each activity has a predecessor and a series of defined successors and is designed according to each application. It is composed of a series of rules that drive the player’s evolution.

Operational Safety Training for Facilities and Laboratories

This application explores the use of BIM of facilities, factories, and laboratories to develop realistic environments for training on machine and equipment operation and safety. In this example, the BIM model of the Technological Center for Industrial Automation (CTAI) at Pontificia Universidad Javeriana is used. This lab has a CIM composed of a dual arm robot with 15 DOF, a CNC machine, conveyors, Automatic Storage, and Retrieval Systems (ASRS), and other industrial machines, as is presented in figure 13.

The goal of this training software is to instruct in the correct operation sequence of the machines, as well as the safety precautions that must be followed to avoid accidents. A complete BIM model of the first level of the CTAI has been developed on Autodesk Revit. The model includes a realistic description of the main machines of this facility. The BIM model is then processed following the workflow described in the previous s

The goal of this training software is to instruct in the correct operation sequence of the machines, as well as the safety precautions that must be followed to avoid accidents. A complete BIM model of the first level of the CTAI has been developed on Autodesk Revit. The model includes a realistic description of the main machines of this facility. The BIM model is then processed following the workflow described in the previous section to be used on the game machine as the model simulation asset.

The gameplay is created defining the competencies and assessment for this training. Competencies include the user being able to identify the main machines and equipment of the laboratory. Also, the user must understand the operational workflow of the CIM, and finally, the safety rules that must be followed to avoid operator injuries.

Assessment focuses on time employed to achieve the full operation of a specific machine or CIM system, the number of injuries, and equipment damage caused by the incorrect operation.

Different flowcharts have been defined for a specific machine. For example, to work safely with the Dual Arm Robot, an operator must follow a specific protocol that includes being in physical contact with the robot controller interface (also known as Tech Pendant), which then unlocks the emergency button, followed by pressing the ‘Servos On’ button. After that, the ‘Start’ button must be pressed for the robot to start working (animation) as is depicted in figure 7.

Injuries on the operator are defined in a similar fashion. In the case of the dual-arm robot, the operator must be outside of the working area of the arms (the work envelope of the robot). In this case, if the operator character is in touch with the robot while it is running the working animation, an accident state is defined, and the injured operator (animation), as is presented in figure 8.

Levels are defined according to the number of machines or elements that must be operated. In the first level, only one machine is operated. The final level includes the operation of CIM (Dual Arm Robot, conveyor, ASRS, and CIM machine), in a space with students’ avatars moving randomly throughout the lab.

Medical Imaging Interpretation

This application focused on generating skills to interpret medical imaging. Because the axial plane is the standard imaging plane used in radiology, data from CT scans and MRI images are usually acquired by the scanner in the axial (or transverse) plane. From this axial dataset, multi-planar reformats (MPR) can be used to view the same information from the coronal and sagittal planes. The radiologist must interpret these different points of view relative to the body to form a tridimensional mental relationship with the real bone structure of the body. Usually, radiology students have difficulty achieving this 2D relationship with the 3D model. This game uses the model obtained from DICOM images to generate a basic 3D cubic block puzzle that students must solve according to a reference image (DICOM) proposed at each stage. The cubic block puzzle is generated by making symmetric planar partitions of the 3D model to generate a collection of smaller cubes that can be assembled to form the full model. The most basic puzzle is generated only using four cubic blocks to be assembled. Figure 9 shows a cranium puzzle composed of 27 cubic blocks.

In the serious game, a DICOM image is presented and the user must assemble the 3D puzzle in such a way that it represents the anatomical plane that the medical image is presenting. The puzzle can be solved to form an axial, sagittal, or coronal slice in a similar fashion, as medical images are formed and presented. The puzzle can also be solved to construct a full 3D model. In the game, the user has a tridimensional grid or matrix in which the puzzle must be solved. Each smaller cube can only be inserted and aligned in one predefined cell of the matrix, although this varies with each puzzle. Only unitary cell size displacements and three axes orthogonal rotations are allowed for each cube. Game progression is based on the difficulty of the puzzle according to the number of blocks to assemble and the reference DICOM image to represent. For this project, only cubic blocks are used to form the puzzle.

The game can be adapted to use different block structures commonly found on educational tools for developing spatial reasoning skills by using other kind of 3D cubic puzzles based on poly-cube packing problems games [29], [30], like Soma cube, Bedlam cube, Diabolica cube, Slothouber-Graatsma puzzle, or Conway puzzle. Figure 10 shows the same cranium puzzle created using the Slothouber-Graatsma puzzle. It is a packing problem that calls for packing six 1 x 2 x 2 blocks and three 1 x 1 x 1 blocks into a 3 x 3 x 3 box. The solution to this puzzle is unique and is a common block puzzle used in education.

Decoration

Decoration is the process of using certain rules created to motivate and involve the player [6]. It includes giving personality to the player avatar and main game objects. It also includes animations that do not affect the game development—but that do immerse the player—making the game fun and engaging. Because proposed games are intended to use the first-person game view, few animations are used with the game avatar. Instead, major efforts focus on providing a full immersive sensation and delivering a virtual 3D model animation experience.

To exploit the CAVE immersion capabilities, the open source UniCAVE prefabs for Unity [31] are used. The UniCAVE project aims to build a plugin for Unity3D that provides support for CAVE environments utilizing features built into the engine. This approach enables users to configure their systems entirely in the Unity3D editor, providing a simplified method for supporting and adapting existing Unity projects for distributed visualization platforms. It also ensures a well-distributed projection on the projection walls of the CAVE, avoiding the operational issues described in [32], that can affect the effectiveness of immersive educational systems.

Figure 11 shows the general structure of the simulation (gameplay) based on 3D objects (.BIM or .OBJ) that have been implemented in this work.

The implemented approach is depicted in figure 12. Model simulation is the core of the serious game and is based on the 3D model. It is a set of related components, scattered around the different game entities. One GameObject can aggregate different types of components: animation, sound, domain simulation, rendering, etc. Each game level uses the same 3D model simulation because learning is focused on achieving a progressive task in the same scenario. A Human Machine Interface (HMI) enables the learner/player to interact with the system and/or with the model simulation. In this case, the skeleton’s gesture obtained using FAAST and Microsoft® Kinect sensor is the mechanism used to interact with the system. Sensorial feedback is sent back to the user. In this case, the CAVE system and the active shutter stereo glasses give a full immersive feedback experience to the user. Every slice along the time axis represents the game state at a defined time or game level. The flowcharts are the managing mechanism for the user’s progression as well as user interactions and model animations. The learning progresses at the same time as game levels increase.

Training Virtualization Serious games Prototypes

In this section, the GameObject architecture of the proposed serious games is presented. Both games, focused on different fields, have been developed using the methodology described in previous sections. The first game is a system for training in operations and safety in facilities, factories, or laboratories. The second game explores the use of this methodology for medical imaging interpretation. This section discusses the game operations in the immersive environment.

Implementations

As described in section ”Model description and game object”, a BIM model of the CTAI facility has been used to create the serious game for CIM safety training operations. Lab construction drawings, as well as machine realistic models, were developed for this model and were exported to the game engine. Users can navigate the virtual world and interact with the defined models. Figure 13 shows a comparison between the real CIM lab and the virtual world used for training. In this game, the 6DOF dual-arm robot, CNC turn, ASRS, and conveyor have been defined as interactive models. According to rules previously defined, the model executes predefined animations according to user activity.

Once the training game launches, a start menu is presented showing the basic interaction commands, a task to execute, and a local mini map to navigate in the game space. A mission checklist and mini map can be displayed at any time during gameplay. The mini map also allows the user to change rooms and helps with navigation. The developed serious game has been tested using the CAVE. The user interacts with the immersive game using a Microsoft® Kinect, tracking the user’s upper arm movements (using relative skeleton joints) as well as gamepad keys. Figure 14 shows some snapshots of the proposed game displayed on the CAVE.

The Medical Imaging training game has been implemented using the puzzle blocks generated from the DICOM model. The puzzle blocks are randomly presented on a stretcher. Segregated s is already on the corresponding cell, allowing the student to continue forming the puzzle according to the defined task. Figure 15 shows the cranium puzzle (medical imaging game) to be solved on a space emulating two different medical rooms. One is a simple scenery and the other is a fully decorated medical facility.

The Medical Imaging interpretation game also has been tested in the CAVE. In this game, through the FAAST interface, students can interact (translate and rotate) the cubic blocks that form the puzzle to be assembled. A DICOM image from one of the anatomical planes is presented and the user must represent this slice using the cubic blocks available on the game surface. Figure 16 shows the proposed Medical Imaging interpretation game working on the CAVE.

Although at this state, only one player has full game control through the Kinect sensor, the CAVE allows other users to simultaneously perceive the game immersion and have direct interaction with the main player (giving instructions and feedback). This direct interaction contributes to game attractiveness and accessibility as proposed in [21]. FAAST middleware allows up to 4 users to be recognized as different skeletons for game control. So, in the future, it is possible to implement games with multiple user control, improving the users’ experience.

Although there are several architectures [5], [33] that support the reuse of the technology components of serious games across different programming languages, game engines, and game platforms, usually they require bridge scripting between programming languages and game engines to allow asset interoperability. In this work, we do not focus on platform interoperability. Instead, we show a methodology that allows integrating professional 3D models in order to create suitable game objects usable by the same game engine. Using the same engine allows a gameplay that reuses modules like user interfaces and the decision trees (or logic rules) defined for game evolution. This replicable methodology allows a faster integration of detailed 3D models with immersive environments to generate vocational training serious games.

Perceptual Evaluation

Video games release versions usually are evaluated using perceptual questionnaires like the System Usability Scale SUS [34] and the Game Engagement Questionnaire GEQ [35]. These evaluation methods focus on measuring how usable is an interface or the player involvement into the gameplay. These instruments have a greater number of questions in order to make an accurate approach to the individual evaluating each of the phases analyzed during game usage (core, in-game, post-game, and social presence modules, among others). Given that our work does not aim towards a final serious game release, we used a set of basic questions similar to those presented by Mihajlovic [36], allowing a fast measurement of the players’ general perception for this kind of training systems.

Both games have been evaluated using focus groups. It is possible to define the minimum sample size for the focus group that guarantees conclusive results as Equation (1) shows:

where,

Z ₁- is a standardized normal distribution with a probability of 1 - ;

α is the confidence used, for this case it is given by the study conducted in [34];

e_m is the mean error of the measurement, that is, the precision; and,

σ _x is the standard deviation of the data obtained.

The acceptable average error chosen for the realization of the tests was ten units. This factor was defined considering the current lecture sizes (below 20 students). According to this factor, the minimum size of the focus group is 5 people. The first group included technicians who had already worked with the CIM system, as well as students taking regular lectures, which includes practice with the CIM and did not have previous knowledge of system operation. The second focus group included students under instruction to work with medical imaging. The age in the technician’s group is between 28 and 40 years. This group is composed of 8 individuals.

On the other hand, the age in the group of students is between 18 and 22 years. The group that works with CIM systems is composed of 20 students. The group that works with the MI game is composed of 11 students.

A perceptual study was performed, asking the users, according to the aforementioned focus groups, to evaluate their experiences with each of the implemented serious games. The users were asked a series of questions with the answers recorded on a 5-point Likert-type scale. The questions followed a similar methodology as the one proposed in [36] and were as follows:

• (Q1) How attractive was the activity? (1-not attractive at all, 5-highly attractive);

• (Q2) How realistic was the serious game environment? (1 = not realistic at all, 5- highly realistic);

• (Q3) Did you feel like you were performing a physical activity? (1-not at all, 5-totally);

• (Q4) Is it likely that you would perform the activity again? (1-not likely at all, 5- very likely);

• (Q5) Do you find the virtual serious game better or worse for learning? (1-worse, 5-much better);

• (Q6) Do you have previous experience with virtual reality or immersive video games? (1-Not at all, 5-High experience).

The individual response distributions for each question are shown in figures 17 and 18.

According to the users’ perception, an immersive serious game for training is an engaging and interesting activity, considering that the virtual environment experience is realistic. In general, a serious game displayed in a virtual environment can be an effective tool for learning. However, as question 1 shows, this kind of tool is more attractive to users without experience in the specific training field, but not to those who already have experience with virtual reality or immersive video games. The latter group tends to be less impressed with this type of development. Question 2 shows that users who already have some knowledge of the training field do not think that the system is highly realistic because it cannot emulate all possible behaviors of the physical system, but those without any previous knowledge have a better perception of game realism. Therefore, this kind of games are more suitable for introductory training rather that advance formation. Question 3 demonstrates that user do not tend to felt as performing a physical activity because movement is limited by the size of the CAVE and the input command used for displacement on the game environment. This limitation can be improved using omnidirectional treadmills for VR environments. As questions 4 to 6 show, this kind of learning tools have great acceptance on persons that have previously worked with this kind of systems because they feel comfortable with this kind of technologies, accepting it as a tool for learning. As question 4 shows, there is large probability for users to perform again a similar activity, however it is necessary to include additional tasks or levels in order to make it attractive again to previous users, according to oral surveys performed after tests. According to question 5, the use of this kind of technology is a novel methodology that can be used on formal training, however, as any educative tool it has to be constantly improved and updated in order to keep the students motivated and involved during the learning process. This perceptual study shows that this kind of VR serious games are an engaging and alternative way to traditional formal training. Additional surveys can be performed in the future to quantify the usability and immersion using protocols like System Usability Scale and Game Engagement Questionnaire, which are commonly used to evaluated commercial technological developments at large scale.

Conclusions

This paper illustrates the use of serious games designed for immersive environments in two different application fields. The first applications focused on machine operations and safety training in engineering facilities and laboratories. The second looks to generate skills in interpreting medical imaging and generating relationships between 2D images and 3D medical models.

The paper presents an approach that can be used to develop different kinds of applications that exploit the advantages that serious games and immersive environments have to achieve learning during a training process. The approach adapts methodologies found in the literature to define a series of stages to develop this kind of training game. It includes the definition of learning objectives, describes the game object building from 3D models, the user interaction, problems, and progressions, as well as game decoration and deployment.

We also have presented a replicable methodology to integrate 3D models for several fields in the development of serious games. BIM files created for architecture and construction purposes can be used to generate realistic 3D games suitable for first-person view navigation and interaction. In a similar fashion, 3D models generated from Medical Imaging are integrated into virtual worlds to generate games focused on skill development. These 3D serious games are complemented by the use of immersive virtual environments —CAVEs— allowing for full immersion and an interactive experience for several users working simultaneously on the same game. The proposed methodology has demonstrated the capability to integrate professional 3D models in order to create suitable game objects usable by the same game engine. Using the same engine, allows module reusage for a faster development for different training fields.

Perceptual evaluation demonstrates that these kinds of tools are well accepted by people familiarized with these technologies and without knowledge in the training area. In this context, young people natively adopt these tools, becoming an alternative to traditional training on complex environments. For trained personal, the system is an important tool for an introductory level. In order to motivate them, it requires more realism or to include non-conventional situations or scenarios like emergencies or unexpected situations.

Although this kind of VR systems can be considered novel for professional training, they need to be constantly updated in order to maintain students involved and focused on the learning process. As any other pedagogical tool, it requires to be properly exploited and complemented with other traditional strategies in order to speed up the learning process.

References

Notes

* Research article

Author notes

^a Corresponding author. E-mail: imondragon@javeriana.edu.co

Additional information

How to cite this article: I. F. Mondragón-Bernal, “A system for immersive medical and engineering training based on serious games,” Ing. Univ., vol. 24, 2020 [Online]. https://doi.org/10.11144/javeriana.iued24.sime

Descargar