Improving productivity and worker conditions in assembly: Part 1 - A collaborative architecture and task allocation

collaborative architecture and task allocation

Conference Paper · October 2018

CITATIONS

0

READS

40

16 authors, including:

Some of the authors of this publication are also working on these related projects:

OPSORO View project

SBO Yves View project Greet Van de Perre Vrije Universiteit Brussel 25PUBLICATIONS   128CITATIONS   

SEE PROFILE

Ilias El Makrini Vrije Universiteit Brussel 11PUBLICATIONS   26CITATIONS   

SEE PROFILE

Bram Boris Van Acker Ghent University 7PUBLICATIONS   15CITATIONS    SEE PROFILE Jelle Saldien Ghent University 68PUBLICATIONS   616CITATIONS    SEE PROFILE

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain

Improving productivity and worker conditions in assembly

Part 1: a collaborative architecture and task allocation framework

Greet Van de Perre

, Ilias El Makrini

, Bram B. Van Acker

, Jelle Saldien

,

Cristian Vergara

, Liliane Pintelon

, Peter Chemweno

, Rapha¨el Weuts

, Karen Moons

,

Reginald Dewil

, Dirk Cattrysse

, Sofie Burggraeve

, Jo˜ao Costa Mateus

,

Wilm Decr´e

, Erwin Aertbeli¨en

and Bram Vanderborght

Abstract— Collaborative robots, or cobots, can improve the working conditions of human operators by decreasing their workload. By assisting the operator both mentally and phys-ically, the overall performance and human wellbeing can be enhanced. The use of cobots in factories introduces new chal-lenges. In this paper, we briefly highlight our contributions to this field. Firstly, we present our collaborative architecture for human-robot assembly tasks, whereby the operator is assisted and guided during the assembly process to lower his cognitive and physical load. In addition we discuss the working principles of our task allocation framework, based upon agent capabilities and ergonomic measurements, and which can be used within the pre-mentioned architecture.

I. INTRODUCTION

For ageing workers, the regular work duties can become burdensome. With age, the physical capabilities gradually decrease, leading to a higher risk of musculoskeletal deficien-cies for physically demanding jobs. In addition, the cognitive capacities such as working memory, patience and flexibility may deteriorate with age, which negatively influences the operator’s performance [1][2]. Especially for this population, the use of cobots can be interesting to decrease both the cognitive and the physical load. While humans have their strength in problem-solving and dexterity, robots comple-ment by their ability of carrying heavy loads and performing repetitive and precise tasks. Therefore, by working together, the robot can assist to lower the physical work load [3]. Additionally, cobots can reduce the operator’s cognitive load to obtain a more qualitative and correct production [4], which is especially interesting for manufacturing in low quantities and high variability.

Human-robot collaboration opens new opportunities in terms of work sharing. Cherubini et al. [5] developed a human-robot manufacturing cell, capable of managing

1 _{Robotics and Multibody Mechanics Research Group, Vrije}

Uni-versiteit Brussel, Belgium, www.brubotics.eu, Corresponding author: Greet.Van.de.Perre@vub.be

2 _{Department of Industrial Systems and Product Design, Faculty of}

Engineering and Architecture, Ghent University, Belgium,

3_{Department of Personnel Management, Work and Organizational}

Psy-chology, Faculty of Psychology and Educational Sciences, Ghent University, Belgium,

4_{Robotics Research Group, PMA Division, KU Leuven, Belgium} 5_{Centre for Industrial Management, KU Leuven, Belgium} 6_{CodesignS, Flanders Make, Belgium}

7_{Flexible Assembly, Flanders Make, Belgium}

human-robot physical interaction and alternating between active and passive behaviors. In [6], robots are put in charge of the quality control in an assembly line. Several task allocation schemes have been proposed to allow human-robot task allocation during collaborative assembly. Tsarouchi et al. [7] proposed an intelligent decision-making method, while in [8], a task allocation framework based on a hierarchical task decomposition is developed. Alternatively, Roncone et al. [9] proposed a transparent task planner capable of allocating tasks and assigning roles.

We here present our collaborative architecture that com-bines face recognition, gesture recognition and human-like robot behavior for enhanced human-robot interaction and visual inspection for quality control during human-robot collaborative assembly tasks. In the developed architecture, the robots role consists in assisting and guiding the human during the assembly process. This is particularly useful when assembling complex parts made of multiple components or for parts with varying designs. This novel system is validated on the collaborative robot Baxter, whereby the robot assists the operator with the assembly of a box. The architecture and validation are briefly discussed in section II.

Additionally, a framework for task allocation in human-robot collaborative assembly was developed, that integrates the agents’ capabilities as well as ergonomics considerations by evaluating the human body posture. A validation of this framework was performed using the cobot Baxter for a collaborative assembly of a gearbox. The framework is discussed in section III.

II. A COLLABORATIVE ARCHITECTURE FOR HUMAN-ROBOT ASSEMBLY TASKS

The developed collaborative architecture is composed of four main modules. The modules for face recognition, ges-ture recognition and human-like robot behavior are used to enhance the human-robot interaction, while the visual inspection module is used for quality control during the assembly process.

A. Social cues for a good collaboration

In order to achieve a good human-robot collaboration, a good communication between human and robot is re-quired. In previous work, we studied how the

commu-obtain a collaboration that reduces the operator’s cognitive and physical load. To identify recommendations to improve collaboration between robots and factory workers, we used the collaborative robot Baxter as a probe to elicit responses from factory workers in order to explore their perceptions regarding working with robots. We explored the willingness of factory workers to work with cobots and investigated which role social cues play. Evidence suggested that factory workers are more willing to work with cobots and enjoy the cobot more when these cobots exhibit more social cues [16][17][18]. Therefore, in our collaborative architecture, the robot uses a number of social components to guarantee a good human-robot communication. Since speech recognition is not possible in noisy environments, gestures are used to interact with the robot. The gesture recognition is performed by a Kinect v2 camera placed on the robot head. The middleware NiTE 2.2 is used to process the Kinect data. The depth, color, IR and audio information of the camera is used to obtain the hand locating and tracking, accurate user skeleton joint tracking and various gesture recognition. In our architecture, the hand waving and the thumbs up gesture are used.

Additionally, face recognition is used for user identifica-tion, allowing to personalize the robot’s behavior for each operator.

A human-like robot behavior is reached by using social cues such as head nodding, head shaking and eye gaze. The gaze is directed to the end-effector of the moving robot arm, which in that way communicates the intention of the robot to the user to increase safety and acceptance. In addition, a head motion system was implemented, consisting of a head nod to indicate acceptance, and a head shaking to indicate refusal. This reactive feedback via the head motion allows the robot to communicate to the operator that it understood his intentions.

To complement user safety aspects while performing col-laborative assembly tasks, a multi-level risk analysis frame-work for hybrid frame-workplace setting is being developed. Based on a task decomposition approach, integrated within hazard analysis, the framework structures safety analysis through generating safety violation scenarios. This essentially allows integrators and designers to assess the sufficiently of safe-guards embedded in the hybrid workplace setting. The risk-analysis framework is currently being tested for a solenoid assembly task.

The information collected upon the risk analysis can be used to direct the level of control the operator needs to have during collaboration with the robot. Default safe states can be defined for the robot system until the operator authorises other states either through direct control or cues.

B. Cognitive support

Research has shown that a major inhibitor for productivity and operator well-being is stress, induced by the complexity

high-variety low-volume. This stress is driven by both physi-cal load as cognitive load [24][25]. Adding hardware in close proximity to the operator can easily increase the complexity (cf. the selection and component task variables determining assembly complexity in [26]) enough to reduce or annihilate the improvements that are targeted by the cyber-physical systems approach. Therefore, it is important to understand the human cognitive reaction and load in such systems. The explanatory framework in [21] can help to contextualize these mental processes and assist measurement procedures and interpretation.

To assist the operator mentally, in the collaborative ar-chitecture, text displays are used to guide the user in the assembly process. Additionally, a visual inspection module is implemented in the architecture, to reduce the prevalence of errors. To avoid error detection only at the end of the assembly, intermediate quality checks are performed during the assembly process. The quality control is carried out by the robot once a new part is assembled by the human. The inherently limited cognitive resources of the human operator [19][20] for, i.a., vigilance, hence become extended with this set of assistive technological aids, optimizing mental workload levels.

C. Architecture validation

To validate this architecture, a human-robot assembly demo of a simple box was implemented in the Baxter robot. The robot guides the assembly by holding and handing the appropriate part. The human task consists in assembling the plate correctly on the semi-assembled box by screwing them. Figure 1 visualizes a set of time instants of the assembly process. Figure 1a depicts the robot holding the semi-assembled box at the appropriate orientation for the assembly of the bottom plate. Figure 1b and 1c respectively show the handing of the plate and the screws. The screwing operation is shown in figure 1d. More detailed information can be found in our previous publication [22].

III. TASK ALLOCATION FOR IMPROVED ERGONOMICS IN HUMAN-ROBOT

COLLABORATIVE ASSEMBLY

In the aforementioned validation of the collaborative ar-chitecture, the sequence and division of the separate subtasks was predetermined. However, to allow a flexible and effective cooperation, a suitable task allocation algorithm should be implemented. Tasks can be assigned to the human or the robot depending on different factors such as capabilities, execution time and performance. We proposed a novel frame-work for task allocation of human-robot assembly applica-tions based on capabilities and ergonomics consideraapplica-tions. Capable agents are determined on the basis of agent charac-teristics and task requirements. Ergonomics is integrated by measuring the human body posture and the related workload.

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain

Fig. 1: Pictures of the collaborative assembly task. In chronological order, the robot is holding the semi-assembled box, handing the plate, the screws to the user and the screwing performed by the operator.

Fig. 2: Framework for the task allocation of assembly tasks. The four modules are represented in bold, i.e. task decomposer, capability evaluator, ergonomics evaluator, and task allocation. The task allocator gathers the inputs from the capability evaluation module and the ergonomics evaluator to assign a given task to an agent. The task executer controls the robot motion while the task translator provides task instructions to the human worker.

A. Framework modules

The developed allocation framework is schematically rep-resented in figure 2 and is composed of four main modules; the task decomposer, the capability evaluator, the ergonomics evaluator, and the task allocation module. In what follows, the different modules are briefly discussed.

The first module of the framework is the task decomposer, which firstly determines the sequence of elementary tasks to be executed using a manipulation taxonomy, whereafter the related task requirements are extracted. To represent the assembly sequence, a diagram is used, which determines the order of the tasks and the different assembly variants. A taxonomy has been developed to classify the manipulation tasks of the assembly sequence. The first classification of the

taxonomy concerns the prehensility of the manipulation task, i.e. if an object needs to be grasped or not. The second step consists in checking if contact interactions exist between the robot and its environment. Finally, the third classification concerns the type of motion, where a distinction is made between the arm motion, namely point-to-point motion and trajectory, and the gripper motion. Using the manipulation taxonomy, the task requirements for the elementary tasks can be extracted.

These task requirements are used as input by the capability evaluation module. Together with the information about the agents and objects to be manipulated, this module determines which agents are capable to perform the subtask.

Necessary information about the agents’ characteristics include data such as payload, speed, reach, equipped grip-per type, and gripping force. Basic objects’ characteristics include weight, dimension, position, and current grasping status. Additionally, the availability of the agents is to be considered for a correct task allocation. An agent might indeed be unavailable, for example when already executing another task.

The ergonomics evaluator module is responsible for as-sessing the physical load during the assembly tasks by evaluating the human body posture. For this, an automatic assessment variant of the well known Rapid Entire Body As-sessment (REBA) method is used [27]. While the traditional REBA technique is an observational pen-and-paper method, we implemented this technique to allow automatic and online estimation of the human posture during interaction tasks. In addition to the calculated REBA score, a value for the workload is determined.

The last module, the task allocator finally allocates the task to one of the agents, based on the inputs from the capability evaluatorand the ergonomics evaluation module. In an operational collaborative work cell, such a task allocation framework would be integrated in a scheduling module that sequences the higher level tasks such that execution time, cost, ergonomics impact, and risk are min-imized. Since multiple jobs and a longer time horizon can be considered, the schedule module can take into account

Furthermore, this scheduling module also covers geometrical allocation of resources, work pieces, operators and robots, since their geometrical location can highly impact the mo-tions of the operator and thus the task allocation, the risk and the ergonomy.

B. Gearbox assembly validation

To validate the task allocation framework, a human-robot collaborative assembly of a gearbox was performed with Baxter. The assembly consists of a number of tasks, including the picking up of parts and tools, inserting them in the right position and fixing screws. The maximum expected REBA scores for every task are determined offline. If one of them succeeds the maximum allowed value of 5, corresponding to a medium risk on MSD, the task is automatically assigned to the robot. The actual, online determination of the REBA value, which is operator and time specific, is used to calculate the cumulative workload. When both the robot and the operator are available agents for a specific task, this value determines the final allocation decision. Figure 3 visualizes three subtasks of the collaborative assembly, together with the calculated REBA score. More information on the task allocation framework can be found in [28].

IV. CONCLUSIONS AND FUTURE WORK Human-robot collaboration can diminish an operator’s workload and improve the working conditions. In this paper, we firstly described our collaborative architecture for human-robot assembly tasks. To obtain an intuitive human-human-robot interaction, human social cues are implemented. The gesture recognition module provides a way to communicate with the robot during the assembly process, while face recognition makes it possible to identify a user. To lower the operator’s cognitive load, a visual inspection module is implemented. Secondly, we presented a framework for task allocation of human-robot assembly tasks that integrates ergonomics con-siderations by measuring the human body posture and taking the geometry of the collaborative workcell into account. Both the cognitive architecture and the task allocation framework were validated through a collaborative assembly task with the cobot Baxter.

ACKNOWLEDGMENT

This work was supported by the SBO project Yves of Flanders Make, the strategic research centre for the manu-facturing industry.

REFERENCES

[1] R. W. Morrell and D. C. Park, “The effects of age, illustrations, and task variables on the performance of procedural assembly tasks.” Psychology and aging, vol. 8, no. 3, p. 389, 1993.

[2] M. S. Young, K. A. Brookhuis, C. D. Wickens, and P. A. Hancock, “State of science: mental workload in ergonomics,” Ergonomics, vol. 58, no. 1, pp. 1–17, 2015.

laboration for bin-picking tasks to support low-volume assemblies,” in Human-Robot Collaboration for Industrial Manufacturing Workshop, held at Robotics: Science and Systems Conference (RSS 2014), 2014. [4] A. Bauer, D. Wollherr, and M. Buss, “Human–robot collaboration: a survey,” International Journal of Humanoid Robotics, vol. 5, no. 01, pp. 47–66, 2008.

[5] A. Cherubini, R. Passama, A. Crosnier, A. Lasnier, and P. Fraisse, “Collaborative manufacturing with physical human–robot interaction,” Robotics and Computer-Integrated Manufacturing, vol. 40, pp. 1–13, 2016.

[6] A. Banh, D. J. Rea, J. E. Young, and E. Sharlin, “Inspector baxter: The social aspects of integrating a robot as a quality inspector in an assembly line,” in Proceedings of the 3rd international conference on human-agent interaction. ACM, 2015, pp. 19–26.

[7] P. Tsarouchi, A.-S. Matthaiakis, S. Makris, and G. Chryssolouris, “On a human-robot collaboration in an assembly cell,” International Journal of Computer Integrated Manufacturing, vol. 30, no. 6, pp. 580–589, 2017.

[8] L. Johannsmeier and S. Haddadin, “A hierarchical human-robot interaction-planning framework for task allocation in collaborative industrial assembly processes,” IEEE Robotics and Automation Letters, vol. 2, no. 1, pp. 41–48, 2017.

[9] A. Roncone, O. Mangin, and B. Scassellati, “Transparent role assign-ment and task allocation in human robot collaboration,” in Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE, 2017, pp. 1014–1021.

[10] C. Vergara, G. Borghesan, E. Aertbeli¨en, and J. De Schutter, “In-corporating artificial skin signals in the constraint-based reactive control of human-robot collaborative manipulation tasks,” in 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM), Singapore, 2018, p. Forthcoming.

[11] E. Aertbeli¨en and J. De Schutter, “eTaSL/eTC: A constraint-based task specification language and robot controller using expression graphs,” in Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. Chicago, IL, USA: IROS2014, 2014, pp. 1540–1546.

[12] E. Aertbeli¨en and J. De Schutter, “Learning a predictive model of human gait for the control of a lower-limb exoskeleton,” in 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics, 2014, pp. 520–525.

[13] M. E. Tipping and C. M. Bishop, “Probabilistic principal component analysis,” Journal of the Royal Statistical Society (Series B), vol. 61, no. 3, pp. 611–622, 1999.

[14] J. Baeten, H. Bruyninckx, and J. De Schutter, “Integrated vision/force robotics servoing in the task frame formalism,” The International Journal of Robotics Research, vol. 22, no. 10, pp. 941–954, 2003. [15] P. Mittendorfer, Eiichi, and Cheng, “3d surface reconstruction for

robotic body parts with artificial skins.” Vilamoura, Portugal: IROS2012, 2012, pp. 4505–4510.

[16] B. Elprama, I. El Makrini, and A. Jacobs, “Acceptance of collaborative robots by factory workers: a pilot study on the importance of social cues of anthropomorphic robots,” in International Symposium on Robot and Human Interactive Communication, 2016.

[17] S. A. Elprama, C. I. Jewell, A. Jacobs, I. El Makrini, and B. Vander-borght, “Attitudes of factory workers towards industrial and collabora-tive robots,” in Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 2017, pp. 113–114.

[18] I. El Makrini, S. A. Elprama, J. Van den Bergh, B. Vanderborght, A.-J. Knevels, C. I. Jewell, F. Stals, G. De Coppel, I. Ravyse, J. Potargent, et al., “Working with walt; how a cobot was developed and inserted on an auto assembly line,” IEEE Robotics & Automation Magazine, 2018.

[19] C. D. Wickens, “Multiple resources and performance prediction,” Theoretical issues in ergonomics science, vol. 3, no. 2, pp. 159–177, 2002.

[20] ——, “Multiple resources and mental workload,” Human factors, vol. 50, no. 3, pp. 449–455, 2008.

[21] B. B. Van Acker, D. D. Parmentier, P. Vlerick, and J. Saldien, “Understanding mental workload: from a clarifying concept analysis

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain

Fig. 3: Pictures of the collaborative gearbox assembly for three different tasks with the associated REBA scores.

toward an implementable framework,” Cognition, Technology & Work, pp. 1–15, 2018.

[22] I. El Makrini, K. Merckaert, D. Lefeber, and B. Vanderborght, “Design of a collaborative architecture for human-robot assembly tasks,” in Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on. IEEE, 2017, pp. 1624–1629.

[23] A. Brolin, P. Thorvald, and K. Case, “Experimental study of cognitive aspects affecting human performance in manual assembly,” Production & Manufacturing Research, vol. 5, no. 1, pp. 141–163, 2017. [24] J. E. McGrath, “Stress and behavior in organizations,” Handbook of

industrial and organizational psychology, vol. 1351, p. 1396, 1976. [25] G. Matthews, S. E. Campbell, S. Falconer, L. A. Joyner, J. Huggins,

K. Gilliland, R. Grier, and J. S. Warm, “Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry.” Emotion, vol. 2, no. 4, p. 315, 2002.

[26] M. Richardson, G. Jones, M. Torrance, and T. Baguley, “Identifying the task variables that predict object assembly difficulty,” Human factors, vol. 48, no. 3, pp. 511–525, 2006.

[27] s. Hignett and L. McAtamney, “Rapid entire body assessment (reba),” vol. 31, pp. 201–205, 2000.

[28] I. El Makrini, K. Merckaert, J. De Winter, D. Lefeber, and B. Van-derborght, “Task allocation for improved ergonomic in human-robot collaborative assembly,” 2018.

Workshop on Robotic Co-workers 4.0: Human Safety and Comfort in Human-Robot Interactive Social Environments 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain