Human-Focused Digital Twin Applications for Occupational Safety and Health in Workplaces: A Brief Survey and Research Directions

Abstract

Occupational safety and health is among the most challenging issues in many industrial workplaces, in that various factors can cause occupational illness and injury. Robotics, automation, and other state-of-the-art technologies represent risks that can cause further injuries and accidents. However, the tools currently used to assess risks in workplaces require manual work and are highly subjective. These tools include checklists and work assessments conducted by experts. Modern Industry 4.0 technologies such as a digital twin, a computerized representation in the digital world of a physical asset in the real world, can be used to provide a safe and healthy work environment to human workers and can reduce occupational injuries and accidents. These digital twins should be designed to collect, process, and analyze data about human workers. The problem is that building a human-focused digital twin is quite challenging and requires the integration of various modern hardware and software components. This paper aims to provide a brief survey of recent research papers on digital twins, focusing on occupational safety and health applications, which is considered an emerging research area. The authors focus on enabling technologies for human data acquisition and human representation in a virtual environment, on data processing procedures, and on the objectives of such applications. Additionally, this paper discusses the limitations of existing studies and proposes future research directions.

1. Introduction

Occupational illness and injuries can result in short-term and long-term lowered quality of life or disability in workers, which can lead to significant costs and economic burdens to households, companies, and governments. The objective of occupational safety and health (OSH) is to minimize hazards and risks within workplaces to prevent occupational illness and injuries, and employers and companies are the ones responsible for OSH in many countries [1]. During the past few decades, OSH has been one of the most challenging issues in practical workplaces. Additionally, in the Industry 4.0 era, some parts of the manufacturing process are expected to become automated, while other parts will still be handled by human workers. Thus, OSH is anticipated to remain challenging in the Industry 4.0 era [2].

Recently, modern technologies, including the Internet of Things (IoT), sensor networks, wireless communication, big data, artificial intelligence (AI), cloud computing, robotics, etc., have emerged as promising tools for improving productivity and cost reduction [3,4]. This megatrend is called Industry 4.0, or the fourth industrial revolution. Such Industry 4.0 technologies enhance competitiveness in a wide range of industries, including manufacturing, construction, and warehouses; however, they can cause new types of hazards and risks [2,5]. For instance, automated facilities and collaborative robots interacting with human workers can cause unintentional contacts and collisions between human workers and physical objects such as machines and robots, which may lead to accidents and injuries [6,7,8]. Thus, OSH should be carefully considered when applying Industry 4.0 technologies accompanying potential hazards and risks. On the other hand, some existing OSH problems can be dealt with more efficiently by applying Industry 4.0 technologies to collect and process various data more quickly and to automate OSH management procedures [9,10]. Consequently, the impacts of Industry 4.0 technologies on OSH are expected to be significant.

This paper focuses on the digital twin (DT), one of the most important Industry 4.0 technologies [8]. DT is a computerized representation in the digital world of a physical asset available in the real world. To reflect a physical asset’s status and characteristics, DT must be integrated and synchronized with the associated physical asset [11,12,13]. Recently, more and more researchers and organizations have attempted to apply DT technology to workplaces for cost reduction, the efficient utilization of resources, improved productivity, and so on [14,15]. Additionally, some researchers have pointed out that DTs can be a novel approach to dealing with various issues related to OSH since they enable objective decision-making based on data analyses and real-time feedback [15,16,17].

Nevertheless, the number of research papers on DT applications for OSH has been fairly limited so far. The main obstacle to developing such applications is probably that the DT system is implemented by integrating several modern technologies [16]. Specifically, DT for OSH should be equipped with software and hardware suitable for collecting and processing data about human workers and for representing and visualizing the status of human workers in the digital world. In other words, DT for OSH is a complex system from the technology perspective since it should focus on human workers. Another issue surrounding DT for OSH is that classical OSH methods, such as checklists and expert assessments, are still widely used in practical workplaces [18,19]. DT technology is expected to be applicable to a wider range of OSH problems, and the benefits of DT for OSH need to be further studied. In these contexts, designing and developing DT applications for OSH are still challenging.

This paper aims to provide practical insights into DT applications for OSH. First, a brief survey of relevant research papers is presented, focusing on the visual representation of human workers. Then, trends and issues in research on DT applications for OSH are analyzed. This paper specifically investigates what kinds of OSH-related objectives can be achieved with DT technology and what kinds of enabling technologies can be applied. Finally, several research directions for future research on DT for OSH are suggested.

The remainder of this paper is organized as follows. Section 2 explains the scope and procedure for research paper collection during the paper survey. Section 3 summarizes the basic results of the paper survey and presents the trends in research on DT applications for OSH. Section 4 investigates the collected research papers in more detail, and discusses the objectives, structures, technologies, roles, and limitations of the existing applications. Finally, concluding remarks and future research directions are given in Section 5.

2. Search Procedure of the Paper Survey

Figure 1 depicts the overall survey procedure of this paper. The first step is to search research papers using keywords. A combination of two keywords, keyword1 and keyword2, was used in hopes that relevant research papers would be collected and the keywords are listed in

Table 1. Keywords for the initial search.

Keyword1	Keyword2
Digital twin Cyber physical system	Occupational safety and health Injury Fatigue Workplace hazard Ergonomics

. Keyword1 specifies the technologies that should be utilized and exploited in research papers for our survey. As shown in Table 1, this paper considers two technologies, DT and cyber-physical system (CPS), for keyword1. CPS is also an important Industry 4.0 technology, where a physical asset and the associated DT constitute a closed-loop control system. In other words, DT is used as a controller for the associated physical asset in CPS [11,14]. In this context, DT and CPS are closely related to each other, and both of them are considered for keyword1. On the other hand, five terms related to OSH issues are used as keyword2.

To ensure the quality of the collected research papers, only research papers published in academic journals indexed on the Web of Science (WoS) and written in English were considered in this step. Research papers published in conference proceedings or journals not indexed in WoS were out of the scope of our survey. Note that about 800 research papers were collected in this step.

This paper focuses on research papers satisfying the following three conditions, and other research papers are out of the scope of this paper. Hence, the authors manually checked the searched papers, and a research paper was excluded from our survey if it did not meet one or more conditions in the filtering phase, as shown in Figure 1.

• Specific applications for OSH should be presented.
• Data about human workers should be collected and utilized.
• Visual representation of human workers should be provided.

Condition (1) indicates that only research papers that propose specific applications—DT systems designed and developed to solve an OSH-related problem—were considered in our survey. In contrast, research papers on conceptual frameworks, general guidelines, and literature reviews of DT for OSH were excluded. Moreover, research papers about single components of DT systems, such as sensor units, or DT applications for non-OSH issues such as production scheduling, were also not considered in this paper.

The other two conditions imply that the DT application should be human-focused, which means that a DT for human workers must be provided and visualized. Some research papers proposed that DT applications designed to collect and utilize data about facilities or buildings can be used to prevent injuries and accidents in workplaces [20,21]. However, those papers do not satisfy condition (2). Condition (3) clarifies that this paper emphasizes the visual representation of human workers. Some researchers have pointed out that visual representation is not an integral part of the DT paradigm, where any “digital” representation can be adopted to develop DT applications [12]. Nevertheless, visual representation is emerging as an important element of DT application, since it can be used for monitoring, user interaction, and analysis [22,23]. Thus, only DT applications with a visual representation of human workers were considered in this paper, and simple wearable devices or sensor applications do not satisfy condition (3).

3. Search Results

3.1. Relevant Research Papers

In this paper, an existing paper is called “relevant” if it satisfies the aforementioned three conditions. Only 15 relevant research papers were identified during the filtering phase in February 2023, which means that DT applications satisfying the above three conditions are still an emerging research area and that relevant research papers are scarce yet. The objective of the reference search step in Figure 1 was to explore the references listed in the research papers found in the filtering phase, where ten more relevant research papers were identified. Consequently, our survey was performed using 25 research papers, listed in

Table 2. Relevant research paper list.

No.	Research Paper
1	Vignais et al. (2013) Innovative system for real-time ergonomic feedback in industrial manufacturing [24]
2	Battini et al. (2014) Innovative real-time system to integrate ergonomic evaluations into warehouse design and management [25]
3	Kim et al. (2018) Ergotac: A tactile feedback interface for improving human ergonomics in workplaces [26]
4	Caputo et al. (2019) Digital twins to enhance the integration of ergonomics in the workplace design [27]
5	Kanazawa et al. (2019) Adaptive motion planning for a collaborative robot based on prediction uncertainty to enhance human safety and work efficiency [28]
6	Nikolakis et al. (2019a) The digital twin implementation for linking the virtual representation of human-focused production tasks to their physical counterpart in the factory-floor [14]
7	Nikolakis et al. (2019b) A cyber physical system (CPS) approach for safe human–robot collaboration in a shared workplace [29]
8	Oyekan et al. (2019) The effectiveness of virtual environments in developing collaborative strategies between industrial robots and humans [30]
9	Akanmu et al. (2020) Cyber-physical postural training system for construction workers [31]
10	Bortolini et al. (2020) Motion analysis system (MAS) for production and ergonomics assessment in the manufacturing processes [32]
11	Greco et al. (2020) Digital twin for monitoring ergonomics during manufacturing production [17]
12	Menychtas et al. (2020) Analyzing the kinematic and kinetic contributions of the human upper body’s joints for ergonomics assessment [33]
13	Zhang et al. (2020) Recurrent neural network for motion trajectory prediction in human–robot collaborative assembly [34]
14	Dimitropoulos et al. (2021) Seamless human–robot collaborative assembly using artificial intelligence and wearable devices [35]
15	Jiang et al. (2021) Cyber physical system for safety management in smart construction site [36]
16	Liu and Wang (2021) Collision-free human–robot collaboration based on context awareness [37]
17	Lv et al. (2021) A digital twin-driven human–robot collaborative assembly approach in the wake of COVID-19 [38]
18	Maruyama et al. (2021) Digital twin-driven human robot collaboration using a digital human [39]
19	Zhao et al. (2021) IoT and digital twin enabled smart tracking for safety management [13]
20	Battini et al. (2022) WEM-platform: A real-time platform for full-body ergonomic assessment and feedback in manufacturing and logistics systems [40]
21	Choi et al. (2022) An integrated mixed reality system for safety-aware human–robot collaboration using deep learning and digital twin generation [4]
22	Gualtieri et al. (2022) Development and validation of guidelines for safety in human–robot collaborative assembly systems [8]
23	Sharotry et al. (2022) Manufacturing operator ergonomics: A conceptual digital twin approach to detect biochemical fatigue [15]
24	Simonetto et al. (2022) A methodological framework to integrate motion capture system and virtual reality for assembly system 4.0 workplace design [41]
25	Caterino et al. (2023) Digital ergonomics: An evaluation framework for the ergonomic risk assessment of heterogeneous workers [42]

.

3.2. Demographics of Relevant Research Papers

Figure 2 illustrates the number of relevant research papers by year, where the following observations can be made: First, no research papers on human-focused DT for OSH had been published before 2013, which indicates that both concepts of DT and its technological enablers were immature in the early 2010s, since the vision of Industry 4.0 was suggested in 2013 [12,16]. Second, most of the relevant research papers (22 out of 25) were published in the last four years, while only three papers were published from 2013 to 2018. Increasing interest is being paid to human-focused DT for OSH, and a more comprehensive range of such applications has become feasible owing to recent modern information and communication technologies (ICTs). Moreover, this trend is similar to those reported in recent surveys on DT [43,44].

Figure 3 shows the geographical distribution of the research papers, where the nationality of a research paper is determined according to the location of the first author’s affiliations. Only three continents, Europe, Asia, and North America, have one or more relevant research papers, which implies that human-focused DT for OSH is still an emerging research topic. Among the three continents, human-focused DT for OSH is most actively studied in Europe, and 60% of the research papers (15 out of 25) were authored by first authors from Europe. Publications from Asia and North America account for 28% and 12% of the relevant research papers, respectively.

In the WoS database, a journal can be associated with one or more research areas, and Figure 4 shows the five most frequent research areas of the journals that published relevant research papers. Note that a single research paper can be associated with two or more research areas.

Computer science is the most frequent research area for publications about human-focused DT for OSH. To implement a DT application, several modern software and hardware technologies must be applied and integrated, and such technologies are typically studied by researchers from the field of computer science [16]. Computer science is essential in implementing human-focused DT for OSH.

Industrial engineering is the second most frequent research area. Indeed, OSH has been one of the most important research topics for ergonomics, a sub-area of industrial engineering. Thus, industrial engineering seems to provide a theoretical basis and practical tools for human-focused DT for OSH [18]. Moreover, the fact that DT is related to theories, methodologies, and technologies from multiple research areas, including computer science and industrial engineering, led to interdisciplinary applications being the third most frequent research area.

Manufacturing engineering and robotics are the fourth and fifth most frequent research areas, respectively. The former shows that manufacturing is OSH’s most popular application domain of human-focused DT. Smart factory, or smart manufacturing, is a critical component of the Industry 4.0 agenda and is implemented by applying modern ICT equipment and infrastructures to manufacturing shopfloors [45,46,47,48]. Thus, measurement, monitoring, and data analysis procedures for occupational illness and injury prevention are expected to become automated and digitalized in a smart factory environment [19,49]. The latter implies new sorts of risks in smart factories. An industrial robot is widely used to automate manufacturing and material handling procedures in a smart factory; however, manual operation by human workers is still an integral part of manufacturing systems [12,15,49,50,51,52]. Consequently, human–robot collaboration (HRC), a collaboration between human workers and industrial robots, will be more and more prevalent in Industry 4.0 [35,53,54]. Since HRC represents a new mechanical type of safety threat, such as a collision between human workers and industrial robots, it can be another exciting application domain for human-focused DT for OSH [4,8].

4. Features and Issues of Existing Applications

In this section, the relevant research papers are investigated in more detail, in order to analyze the features and issues of existing human-focused DT applications for OSH within workplaces.

The first subsection discusses what the existing applications are for, by analyzing their target industries, physical assets, application scenarios, functional objectives, etc. In contrast, the other three subsections are dedicated to important components of these existing applications. With the increasing interest in DT, a number of researchers have proposed reference models or architectures for DT applications, where important components and functionalities are specified [16,22,55,56,57,58]. Different reference models or architectures have different contents; however, there are three commonly identified modules: data collection, digital representation, and analysis and feedback [15].

As mentioned earlier, this paper focuses on applications that collect digital data about human workers and provide visual representations for them. Thus, devices, technologies, and tools for collecting digital data about human workers and rendering their visual representation will be investigated in this section. Data collection and digital representation modules are required to create a DT for a physical asset itself. However, this is not the ultimate goal of the DT application. A DT application should enable the users to make decisions related to workplace operations management more systematically. In this context, the analysis and feedback module is also an essential part of DT application, and a wide range of approaches, from domain knowledge to advanced techniques such as simulation and AI, can be applied to develop this module [4,16,49]. The features of the analysis and feedback modules of the existing applications are also investigated in this section.

4.1. Objective, Application Domain and Test Environment

Figure 5 classifies the objectives of existing human-focused DT applications for OSH into three categories proposed by the authors: ‘Risk Assessment’, ‘Real-time Accident/Injury Prevention’, and ‘Monitoring’. ‘Risk Assessment’ is the objective of about two thirds of existing solutions. These solutions have in common that they are designed to assess the potential risks within workplaces or operations. Typically, the major concerns of these applications are postural risks that can cause musculoskeletal disorder and injury [14,17,24,25,26,27,31,32,39,40,41,42]. In contrast, some of those applications aimed to assess other types of risk, such as human worker fatigue [15,33] and collision risks [8,36]. The assessment results of the applications for ‘Risk Assessment’ can be used to redesign workplaces or manual operations.

‘Real-time Accident/Injury Prevention’ is the second most popular objective in Figure 5. The applications for this objective primarily concern a mechanical type of risk, such as a collision between human workers and robots [4,28,29,30,34,37,38]. Dimitropoulos et al. [35] also considered human workers’ fatigue. To prevent accidents and injuries caused by a collision between human workers and robots in real-time, these applications monitor both human workers and robots and sometimes generate commands for the safe control of the robots. For instance, this DT application can instruct robots to stop, adjust their speed, or move away from human workers when a collision risk is identified [29,37]. In addition, Zhao et al. [13] proposed a DT application that supports ‘Real-time Monitoring’, which means that information about human workers and their context is provided in real-time.

Figure 6 shows the number of relevant publications based on target industry (i.e., manufacturing, general, warehouse, and construction). The analysis shows that manufacturing is the most popular application domain for human-focused DT for OSH. Clearly, human-focused DTs for OSH are based on Industry 4.0 technologies, and a smart factory is a key part of the agenda of the Industry 4.0 era. Thus, the manufacturing industry is the most important application domain for such technologies. Similarly, this is also the case for human-focused DT for OSH [4,8,17,24,27,28,29,32,33,34,35,37,38,40,41,42].

The term ‘General’ in Figure 6 indicates that the associated applications can be applied to a wide range of industries [15,26,30,39]. Warehouse [13,14,25] and construction [31,36] are the third and the fourth most popular target industries, respectively, in Figure 6. In other words, they are a less popular application domain than the manufacturing industry. Nevertheless, they are also promising application domains for human-focused DT for OSH in that both are traditionally labor-intensive industries. Still, state-of-the-art technologies have recently been actively introduced.

In Figure 7, the tasks performed by human workers in existing applications are classified into four categories (i.e., assembly, manual material handling (MMH), ad hoc motion, and others). The majority of the applications were designed to create and utilize DT for human workers performing assembly tasks that involve combining a number of parts to obtain an end product [4,8,17,24,27,28,31,32,33,34,35,38,40,41,42]. Most of these applications target manufacturing applications.

The objective of an MMH task is not to make a product but to move an object from one place to another via common actions such as lifting, picking, and placing. Both assembly and MMH tasks contain a series of actions. However, the latter generally has a smaller number of actions and a shorter cycle time. In addition, the repetitive nature of MMH tasks poses ergonomic risks that can cause musculoskeletal disorders and injuries [12,40]. Thus, most existing applications for MMH tasks are used to assess postural risks or the fatigue of human workers [14,15,25,26,39]. Additionally, Oyekan et al. [30] considered preventing collisions between human workers performing these MMH tasks and robots.

In contrast, four research papers considered other tasks for human workers. Nikolakis et al. [29] and Liu and Wang [37] developed DT applications to prevent workplace collision with HRC. However, these applications were applied to human workers performing arbitrary actions rather than specific assembly or MMH tasks. In other words, these research papers focused on the technical demonstration of human-focused DT for OSH. Moreover, the applications developed by Zhao et al. [13] and Jiang et al. [36] allow human workers to perform daily operations, including travel within relatively large workplaces.

Figure 8 shows that only about a quarter of the relevant research papers performed experiments in practical workplaces [13,25,27,34,35,36,42], while the applications in other research papers were studied under laboratory environments. This suggests that the conceptual design and validation of technical feasibilities are still essential matters for human-focused DT for OSH.

4.2. Data Collection

The physical assets from which existing applications collect digital data are summarized in panel (a) of Figure 9, where a single application can collect data from two or more kinds of assets. First, human workers are important physical assets in all of the 25 existing applications, which is quite natural since the scope of our survey is limited to DT applications that utilize digital data collected from human workers.

In addition, nine applications collected digital data from industrial robots [4,8,28,29,30,35,37,38,39]. To be precise, these applications targeted HRC environments where human workers and robots share workspaces. Human workers have the dexterity required to perform delicate tasks, while robots are powerful, fast, and accurate, so they help achieve higher productivity for simple and repetitive tasks. HRC combines these advantages of human workers and robots, an essential part of a smart factory [35,53,54]. Nevertheless, the coexistence of human workers and robots in a single workspace can cause safety issues, such as collisions between them. Therefore, HRC can be a promising application area for human-focused DT.

Some researchers considered other types of physical assets for their applications. For instance, Sun et al. [49] and Jiang et al. [36] thought of facilities as physical assets for their applications. Materials and tools can also be used as physical assets for human-focused DT applications for OSH [14,27,35].

Panel (b) of Figure 9 shows the devices used to collect digital data in existing applications, where wearables and inertial measurement unit (IMU)-based motion capture (MoCap) systems are the most frequently used devices. Wearables are small electronic devices that can be attached to the human body or clothing and typically enable the detection of signals from the human body and provide connectivity to other ICT devices [59]. IMU is a specific sort of wearable device that can measure the kinematic parameters of a human body, such as the external load and joint angles. IMU sensors, as well as wearables, can hinder human workers from performing their job since these sensors must be attached to the human body or clothing. Nevertheless, IMUs have been widely adopted to measure factors related to the posture and motion of humans, due to their accuracy and reliability [60,61].

Depth cameras and Microsoft Kinect are also popular data collection devices for human-focused DT for OSH [4,14,29,30,32,34,35,37,38]. Depth images obtained using depth cameras contain 3D information about how far an object is from the camera. Kinect is a well-known commercial device that provides a depth camera and related software libraries. A 2D depth image can be converted into a 3D point cloud, a set of points scattered in 3D space, or a 3D skeleton that consists of significant joints and connection lines between them, by applying appropriate algorithms. Consequently, a depth camera and Kinect enable the existence, location, and posture of scene objects, including human workers and robots, to be detected. In recent years, depth cameras and Kinect have emerged as valuable data collection tools for ergonomic assessment and healthcare [62,63].

Two research papers utilized optical MoCap systems [15,39]. Typically, optical MoCap systems are used to collect data about the posture of humans using several cameras, which can detect the markers attached to the human body or clothing. A tiny piece of retroreflective material or light emitting diode can be used as a marker, and a marker for the optical MoCap system represents the position of a specific body segment. Therefore, an optical MoCap system can be used to collect data required to create a 3D skeleton and to measure joint angles. However, the markers or marker-attached body suits can be cumbersome to human workers. Moreover, markers can be dropped during the data collection procedure. These limitations should be carefully considered when applying optical MoCap systems.

In addition, four research papers utilized other data collection technologies, such as indoor positioning [13], non-wearable sensors [28,37], and optical cameras [36].

In panel (c) of Figure 9, the types of data collected by existing applications are classified into four categories: posture, location, movement, and biometric. Posture data are typically collected in terms of joint angles obtained from the skeleton of a human body and are utilized in most relevant research papers. The reason why posture data are widely used is that assessing postural risks that can cause musculoskeletal disorder and injury is useful. Moreover, posture data can be used as input parameters for observational methods widely used for ergonomic assessment, such as the Rapid Upper Limb Assessment (RULA), Rapid Entire Body Assessment (REBA), Ovako Working posture Analysis System (OWAS), etc. [64]. Consequently, many existing applications focus on postural risks accompanied by manual work, as shown in Figure 9.

Location data are also collected and utilized in a significant number of existing applications, including both absolute location in the workplace and relative location, which provide the proximity to specific objects such as robots. In particular, safety distance, the distance between a human worker and a robot, is a good indicator of collision risk, and some applications are designed to stop or slow down the robot when safety distance is not maintained [4,28,29,37,39]. Additionally, location data can be used to analyze the motion trajectory of human workers [14,34,38] or to monitor their status [13,36].

Movement data include information about the changes in a human worker’s location, such as speed, acceleration, and direction of movement. Movement data can be used to assess the magnitude of risk [30], to predict motion trajectories [28], to measure joint torque [33], and to detect accidents [13].

Sometimes, biometric information, including blood pressure, heart rate, breathing rate, and skin temperature, is used to assess fatigue and to monitor the status of human workers [40].

4.3. Visual Representation

A visual representation in a simulated environment is not a necessary feature of DT for a physical asset [12]. However, there is no doubt that appropriate visual representation, which can be adopted for monitoring, analyzing, and controlling the associated physical asset, makes DT a more attractive and novel tool. This paper focuses primarily on human-focused DT applications that provide a visualized DT for a human worker. Panel (a) of Figure 10 shows how human workers are visually represented in the existing applications. Note that some research papers did not outline their visual representations in detail, and thus are not considered in this section.

Panel (a) of Figure 10 shows that whole body avatars are the most popular form of visualization, where various body parts such as the head, hands, arms, and legs are represented [8,14,15,17,24,27,30,31,39,40,41,42]. This form of visualization provides the most realistic representation of the human body and the most impressive visual experiences. However, specialized software tools and libraries are needed to create a whole body avatar and to emulate human workers’ motions.

Skeletons are a form of visualization that represent the human body by using major joints and by drawing connecting lines between these joints [4,25,26,32,33,34,35,38]. The structure of the skeleton is suitable for visualizing the positions of joints and joint angles between body segments, which are often used in observational methods to calculate ergonomic or safety indices.

A point cloud is a set of points indicating the locations occupied by physical assets within a 3D space, and is typically created using depth cameras. Since a point cloud can reflect the actual 3D shape of a human worker, it is suitable for accurately calculating the safety distance between a human worker and a robot. A point cloud can be converted into a full body avatar or skeleton by applying object detection or instance segmentation algorithms. However, several applications have provided visual representations in the form of the point cloud itself [28,29,37].

A symbol is another form of visualization, where the whole body of a human worker is represented as a simple figure. For instance, Zhao et al. [13] and Jiang et al. [36] indicated the existence and location of human workers within workplaces using dots or simple icons.

To provide a visual representation for human workers, modern visualization tools that can process or reflect the collected data are required. Unfortunately, detailed descriptions of these visualization tools are missing in some previous research papers. However, from the papers with detailed descriptions, the visualization tools that provide simulated environments can be grouped into three categories: custom applications, simulation modeling tools, and HW vendor-provided tools.

A custom application is an original visualization tool developed by the researchers. Some researchers applied Unity [4,29,30,31] or the Robot Operating System (ROS) [37,40] to develop custom applications. Unity is a game engine widely used to develop 3D games, enabling the convenient creation of a 3D environment and game characters. If a game character can emulate the motions of a human worker, the environment created by applying Unity can be used as a visualization tool for human-focused DT applications for OSH. ROS is an open-source framework for developing applications associated with robots, and it enables the management of a complex network of distributed sensors, devices, and systems. Thus, ROS can help build a visualization environment for DT applications, typically consisting of hardware and software modules. Of note, some custom applications have not been described in detail in associated publications [13,32,38] but, as shown in panel (b) of Figure 10, about half of the existing applications are equipped with custom applications.

Commercial software for simulation modeling and experiments of human motion has been gaining more and more attention recently. Examples include Siemens Tecnomatix and Dassault Delmia Human. Such software supports the building of a simulation model for workplace production processes and human motion. These models can evaluate performance measures or ergonomic indices by performing simulation experiments. Significantly, the motion of a human worker can be simulated by providing the data collected from a workplace, which has led several researchers to use human motion simulation software as a visualization tool [8,17,27,40,41,42]. Despite their high prices, commercial human motion simulation software provides novel ways to visualize and analyze human motion. Furthermore, open-source simulators such as Gazebo can also be used to provide visual representations [35].

Sometimes, DT applications adopt the MoCap system, which is supplied as a package that includes both hardware devices and software tools for processing the collected data. In this case, the visual representations of human workers can be obtained using hardware vendor-provided software.

4.4. Analysis and Feedback

Human-focused DT applications for OSH should support data analysis and feedback helpful for the prevention of occupational injuries and illness. Figure 11 summarizes the sorts of feedback provided by existing applications. An ‘assessment result’ is obtained following an analysis or calculation using the data collected from a workplace, while a ‘measurement result’ indicates the collected data. ‘Robot control’ refers to the associated commands or signals generated by a DT application to control industrial robots if required. ‘Real-time alert and guide’ is a type of feedback containing information about risks and hazards identified in the workplace, and requesting human workers or managers to take proper actions.

‘Assessment result’ feedback is often provided in the form of a report that includes ergonomic or safety indices [8,14,17,24,25,27,30,31,32,33,40,41,42], the stability of a human’s motions [15], and joint torque [26,33]. This feedback can be used to redesign manual tasks and workspaces, and it is frequently provided off-line.

The ‘robot control’ type of feedback aims to control industrial robots more ergonomically and safely. For instance, Nikolakis et al. [29], Zhang et al. [34], Liu and Wang [37], and Choi et al. [4] developed DT applications that stop the robot or reduce its speed when the safety distance between a human worker and a robot becomes unsafe. Kanazawa et al. [28], Zhang et al. [34], Maruyama et al. [39], Liu and Wang [37], and Lv et al. [38] applied DT applications to make robots perform necessary actions, such as delivering the right tools and moving through the right paths at the right time. Those applications generally took both productivity and OSH issues into consideration. In addition, Dimitropoulos et al. [35] studied DT applications that lead human workers to maintain correct posture.

The ‘real-time alert and guide’ type of feedback contains a wide range of information about OSH issues, including body segments that cause bad posture [24,31], guides for work [39], and the occurrence of dangerous situations [13,36]. ‘Measurement result’ feedback includes the location of human workers [13,36] and their biometric information [40].

In Figure 12, important data analysis methods and approaches for human-focused DT applications for OSH are classified into four groups: observational methods, AI/machine learning (ML)/data mining (DM), simulation, and safety distance.

Typically, the objective of observational methods such as RULA, REBA, and OWAS is to assess the risks in manual work by calculating ergonomic or safety indices. To apply observational methods, data about the posture of a human worker and any manual work must be collected and provided. Traditionally, the data for observational methods have been collected by human experts, who observe the manual work performed by a human worker. However, DT can be used to collect data in an automated and real-time manner, and most existing applications can also be used as the data collection tool for observational methods.

Recently, AI/ML/DM algorithms, which can be used to discover useful patterns and knowledge hidden in a large volume of data, have emerged as promising data analysis techniques in a wide range of application domains. Specifically, artificial neural network (ANN) and deep learning (DL) algorithms have been successfully applied to analyze unstructured data such as those from images and videos, which are typically hard to analyze using traditional algorithms. Some existing applications adopted these algorithms to perform object detection [4,35,36] and human pose estimation [31,37,39]. Object detection and human pose estimation are integral parts of human-focused DT applications if data about a human worker are collected in the form of images or videos. Additionally, AI/ML/DM algorithms have been used for human motion trajectory prediction [28,34], task scheduling [38], etc. [13,15].

Simulation is a practical approach for evaluating the performance of complicated systems. Moreover, simulation is known as an important feature of DT and CPS applications. Caputo et al. [27], Nikolakis et al. [14], Greco et al. [17], Battini et al. [40], Choi et al. [4], and Caterino et al. [42] modeled and simulated human motion to assess risks in manual work and to apply certain observational methods. Kanazawa et al. [28] and Jiang et al. [36] utilized simulations to estimate the risk of collisions in workplaces. Additionally, safety distance is a useful approach for detecting collision risk and preventing accidents in HRC workplaces [4,28,29,37].

5. Conclusions and Further Remarks

5.1. Conclusions

DT is one of the most important Industry 4.0 technologies, and is recognized as a key component of a smart factory. However, many DT applications only consider machines or materials as physical assets, and not human workers, due to several obstacles such as the complexity of human behaviors, the technical difficulties of collecting digital data from human workers, and psychological resistance from human workers. However, human workers are the main subject of OSH. Thus, human workers are an important physical asset for DT applications for OSH, and modern Industry 4.0 technologies enable the collection of much digital data from human workers. In this context, the authors believe that human-focused DT application for OSH is a promising research topic for both researchers and practitioners. This paper provides a brief survey on DT applications that concern OSH within workplaces. Specifically, this paper regards visual representation for the human worker as an integral part of human-focused DT applications for OSH. From the survey results, the following conclusions can be drawn.

First, only 25 relevant publications were identified, which indicates that human-focused DT for OSH is a very specific application domain of DT technology. Moreover, Europe accounts for more than half of the relevant publications. In other words, human-focused DT is not yet a popular research topic in some regions. OSH management generally does not yield immediate financial benefit, and specialized hardware and software modules are required to collect digital data from human workers and to visualize them in digital environments. Nevertheless, human workers are still an essential part of smart factories in the Industry 4.0 era, and the number of relevant publications is increasing.

Second, computer science and industrial engineering are the most frequent research areas for journals that publish relevant research papers. In other words, computer science and industrial engineering provide important methodological and theoretical bases for human-focused DT for OSH. This is partly due to the fact that a significant number of existing applications aim to automate data collection and analysis procedures for traditional observational methods. These observational methods, such as RULA, REBA, and OWAS, are generally useful for assessing postural risks by calculating ergonomic or safety indices. Traditionally, the data required to calculate indices are collected and analyzed manually by industrial engineers, which is inconvenient. Modern Industry 4.0 technologies such as sensor networks and MoCap systems can, however, be used to fill this gap. Thus, computer science is also an important research area for human-focused DT for OSH.

Existing applications for automating observational methods have been innovative; however, they do not consider new types of OSH issues, caused by modern manufacturing facilities within smart factory environments. Specifically, DT technology is expected to lead to the creation of novel approaches and applications for preventing risks and hazards caused by robots that can be actively introduced into many workplaces. Thus, robotics is also an important research area for human-focused DT for OSH.

Third, the most popular application domain is assembly tasks in the manufacturing industry. Typically, an assembly task contains a sequence of actions. DT can be used not only to assess postural risks of specific actions, but also to check the progress of the assembly and to provide dynamic guidance and support to human workers. In other words, human-focused DT applications can contribute to both productivity and OSH, and they might be prevalent in future smart factory environments.

Fourth, cameras, including the Kinect system, are becoming popular as data collection devices. In addition, posture data, represented by the positions of joints and joint angles, are the most popular data type for human-focused DT for OSH. By 2018, three relevant research papers were published, and they all used wearables and IMU-based MoCap systems. Though these systems are costly and can annoy human workers during manual tasks, they produce reliable motion data. Moreover, the collected data can be processed in relatively simple ways without complicated algorithms and high computing powers. In contrast, a single camera is generally inexpensive compared with MoCap systems regardless of whether it is a depth type or not, and it does not use any tiny devices or markers that need to be attached to the human body or clothing. Analyzing the image or video data produced by a camera using traditional techniques is difficult, but modern computer hardware and AI algorithms enable pre-processing procedures such as object detection, instance segmentation, and human motion recognition to facilitate these analyses. Consequently, cameras and AI algorithms are expected to be more widely applied to develop human-focused DT applications in the future.

Fifth, a visualization tool must be carefully chosen and planned. For instance, the location of a human worker, represented as a symbol or an icon, can be visualized using relatively simple tools. Such real-time location information can be intuitively understood and can provide additional information, such as the number of human workers in a dangerous area. In contrast, more complicated forms of visualization such as a whole body avatar require more specialized visualization tools. Note that a whole body avatar is the most popular form of visualization for human workers in relevant applications. Developing custom applications that provide good-looking visual representations may be time-consuming and may prolong the entire DT application development period. Hardware vendor-provided software and commercial human motion simulation software are also available for the visual representation of the human workers; however, they can be considerably expensive and hard to customize. Therefore, an appropriate visualization tool is a key component for successfully developing a human-focused DT application for OSH.

Sixth, only about a third of existing applications utilize simulations, though many researchers pointed out that simulation is an important feature of DT applications and CPS. Most of existing applications focus on human motion simulations using commercial human motion simulation software; however, the simulation modeling and experiment procedures are generally not described in detail. One reason for the low popularity of simulation is that traditional observational methods can be applied without simulation modeling and experiments. Moreover, developing a custom application enabling simulation modeling and experiment is non-trivial; commercial simulation software may be a practical alternative, but commercial human motion simulation software is often costly, which is thought to be another cause of the low popularity of simulations.

5.2. Future Research Topics and Challenges

One limitation of this paper is that not many existing publications were relevant, since human-focused DT application for OSH is still an emerging research topic. Nevertheless, the number of relevant publications is increasing, and more and more applications will be studied and introduced in the future. Another limitation is that this paper focused primarily on the technological aspects of relevant applications, whereas legal and ethical issues resulting from human-focused DT applications have largely not been discussed. Based on these limitations and the survey results, this paper suggests the following future research topics and directions.

The first research topic is the development of human-focused DT applications for various purposes related to OSH. While almost all existing applications concern postural risk and collisions between human workers and industrial robots, some other OSH issues can be solved by applying DT technology. Failures and accidents that can lead to illness and injury are caused by various factors such as organizational influences, supervisory factors, preconditions for unsafe acts, and unsafe acts [65]. There is a tendency to blame work personnel for accidents; however, determining the main cause of an accident is difficult [66]. Human-focused DT may thus be used to record and monitor the context of failures and accidents. Consequently, DT can also contribute to finding the main causes of accidents and reducing the blame on human workers involved in accidents.

The second research topic is more simple and convenient data collection devices. With the advance in computer hardware and AI algorithms, cameras have emerged as a useful device for data collection in human-focused DT applications for OSH. State-of-the-art object detection and human motion recognition algorithms enable even low-cost webcams to be used to collect digital data on human workers, and the MoCap system and depth cameras have been widely adopted in existing applications. Human-focused DT applications could thus be implemented more quickly and conveniently if more simple data collection devices are used.

Third, simulation modeling and experiments for purposes other than human motion simulation are another future research topic. Simulation is a tool for comparing the performances of a number of alternatives and for identifying the best one among them. Thus, simulation can be used to optimize a workplace such that risks are minimized. For instance, existing applications concerning HRC workplaces are typically designed to stop or slow down a robot when a collision risk is detected. This is obviously helpful for reducing the risk in a workplace; however, frequently stopping or slowing down a robot can decrease job efficiency and productivity. In this case, simulation can be used to find the optimal facility layout or motion path so that risks are minimized while maintaining job efficiency and productivity.

Finally, some researchers might be interested in legal and ethical issues. Both employers and employees can regard human-focused DT applications that collect digital data from human workers as a means for monitoring human workers. On the one hand, human-focused DT applications might lead to the creation of novel people analytics, which would aim to evaluate human workers’ behaviors and performances in a comprehensive manner. On the other hand, those applications can cause sensitive issues. For instance, the fact that digital data are being collected raises concerns for some human workers because they feel that their employers or supervisors are watching their every move, which can intensify work stress and can erode worker autonomy. In this case, should managers inform workers of what kinds of data are collected? Is data collection without worker consent illegal? How can employers mitigate the resistance of human workers to data collection? Some digital data, such as videos recorded during work, contain personal identifiable information. Data anonymization or privacy protection procedures should thus also be considered when developing applications that utilize such data.