Taking back the wheel: transition of control from automated cars and trucks to manual driving

S T2021/11

Summary

Until automated driving systems are capable of performing all driving tasks under all road conditions, drivers will have to take over control when the automation fails or reaches its operational limits. This thesis tackles challenging Human Factors issues related to control transitions to manual, particularly in automated truck platooning scenarios. The research findings contribute to a better understanding of driver take-over process and the variability between and within drivers.

About the Author

Bo Zhang conducted her PhD research at the Department Centre for Transport Studies, University of Twente, as part of the Marie Curie ITN project ‘HFauto’. She holds a Master’s degree in Human Factors Engineering from Technical University of Munich.

TRAIL Research School

Taking Back the Wheel:

Transition of Control From

Automated Cars and Trucks

to Manual Driving

Bo Zhang

Bo Zhang

Taking Back the Wheel: T

ransition of Control From Automated Cars and T

CONTROL FROM AUTOMATED CARS AND

TRUCKS TO MANUAL DRIVING

Bo Zhang University of Twente

This research was funded by the Marie Curie Initial Training Network (ITN) project HFAuto – Human Factors of Automated Driving (PITN-GA- 2013-605817), TNO, and the Department Centre for Transport Studies, Faculty of Engineering Technology, University of Twente.

Cover illustration by Bo Zhang, inspired by the H-Metaphor that driving automated vehicles is like riding horses (Flemisch et al., 2003).

DISSERTATION

to obtain

the degree of doctor at the Universiteit Twente, on the authority of the rector magnificus,

prof. dr. ir. A. Veldkamp,

on account of the decision of the Doctorate Board to be publicly defended

on Friday 12 February 2021 at 16.45 hours

by

Bo Zhang

born on the 28th of August, 1988 in Heilongjiang, China

CONTROL FROM AUTOMATED CARS AND TRUCKS

TO MANUAL DRIVING

co-supervisor: Prof. dr. ir. E.C. van Berkum

Composition of the doctoral committee: Prof. dr. ir. H.F.J.M. Koopman chairperson Prof. dr. M.H. Martens supervisor Prof. dr. ir. E.C. van Berkum co-supervisor

Independent members:

Prof. dr. L. Boyle University of Washington Prof. dr. K. Bengler Technical University of Munich Prof. dr. M. Hagenzieker Delft University of Technology Prof. dr. M.C. van der Voort University of Twente

Prof. dr. W.B. Verwey University of Twente

TRAIL Thesis Series no. T2021/11, the Netherlands Research School TRAIL

TRAIL P.O. Box 5017 2600 GA Delft The Netherlands E-mail: info@rsTRAIL.nl ISBN: 978-90-5584-284-1 Copyright © 2021 by Bo Zhang

All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without written permission from the author.

Dedicated to

My Mother Yuehua Zhang, who named her daughter ‘Bo (博)’ hoping that she would become a PhD (博士) one day. Your dream now has come true.

献给我的妈妈张月华：

i

Preface

This thesis is the result of a PhD research carried out at University of Twente, Department Centre for Transport Studies from 2016 to 2020. Now this long journey has finally come to an end, filled with treasurable experiences that have shaped me into a qualified scientific researcher, a persevering person, and a critical thinker. Through all the ups and downs I was never alone, and I would like to express my sincere gratitude to all people who made this possible.

First of all, I would like to thank my promotors Marieke Martens and Eric van Berkum. They believed in my potentials and provided me the great opportunity to join their Department and develop my personal and professional skills. I am particularly grateful to Marieke for her all-round guidance and support. Thank you, Marieke, for sharing your invaluable experience and perspective as a successful female scientist, for helping me through the obstacles when the road got bumpy, and for all the inspiring and encouraging talks that lit up my day and kept me going steadily towards completing my PhD. My special thanks also go to Riender Happee and Joost de Winter at TU Delft, who welcomed me into the HFauto family, hosted my secondment in the early stage of my PhD, and provided continuous supervision and mentoring throughout my studies. Thank you Riender for reaching out to me and introducing me to Marieke when I was a fresh master graduate; my journey started because of you. Joost, your high standard of academic research pushed me further than I could imagine, and I will forever benefit from what I have learned from you. I am very grateful to Klaus Bengler for hosting my secondment at my old school TU Munich. Thank you for introducing me to the fantastic realm of Human Factors, and for supporting every important decision in my study and career.

An essential part of my research concerns driver behaviour in automated platooning scenarios, and this could not be completed without generous support from the amazing colleagues at TNO. I am grateful to have participated in the TNO Early Research Program (ERP) Human Enhancement: Adaptive Automation, and to have the access to large amounts of experimental data that formed the base for four chapters in this thesis. I want to express my deep gratitude to Ellen Wilschut, Dehlia Willemsen, and Jeroen Hogema for their invaluable scientific advice and contributions as co-authors, to Ingmar Stel for technical support, and to Bart Joosten for inspiring brainstorm sessions. My thanks also go to Tom Alkim at Rijkswaterstraat for giving me the opportunity to work on the see-through truck study and for revising the manuscript for publication.

Many thanks to Linda Boyle, Klaus Bengler, Marjan Hagenzieker, Mascha van der Voort, and Willem Verwey for being my doctoral committee members. Thank you for your time and patience, and for your intellectual contributions to my development as a scientist.

I would like to thank all my HFauto research fellows for sharing their knowledge and expertise in diverse research areas, and for the fantastic time at the consortium meetings. Many thanks to Silvia Varotto and Zhenji Lu at TU Delft for their invaluable contributions to two important chapters of my thesis, and for their encouragement and companion as friends. Thanks to Marjan Hagenzieker, Christopher Cabrall, Pavlo Bazilinskyy, and Miltos Kyriakidis at TU Delft, Daniel Heikoop, Alexander Eriksson and Neville Stanton at the University of Southampton, Joel Gonçalves, Bastiaan Petermeijer and Klaus Bengler at the TU Munich, Matt Sassman, Thierry Bellet and Marie-Pierre Bruyas at IFSTTAR Lyon, Alberto Morando and Marco Dozza at Chalmers University of Technology, Ignacio Solis, Veronika Petrovych, Katja Kircher, Jan Andersson, and Magnus Hjälmdahl at VTI Linköping. I feel lucky to be part of the big family and to learn from you at the early stage of my training, which helped kick-start my research. I’m also grateful to fellow researchers outside the HFauto consortium, Anna Feldhütter and Andre Dietrich at TU Munich, for supporting the experiment during my joint secondment with Zhenji.

Many thanks also go to my colleagues at the Department Centre for Transport Studies, University of Twente: Amelia Huang, IG Ayu Andani, Anika Boelhouwer, Francesco Walker, Tom Thomas, Kasper van Zuilekom, Mariska van Essen, Oskar Eikenbroek, Karst Geurs, Lissy La Paix Puello, John Pritchard, Tiago Fioreze, and others. Thank you for all the good time we spent together and for the inspiring discussions during lunch and coffee breaks. Many thanks to our secretary Dorette Olthof for making me feel at home from day one.

My life would be pale and I would never come this far without wonderful friendship developed along the way. Special thanks go to my paranymphs Ellen Wilschut and Silvia Varotto. Ellen, thank you for your strong support in every possible aspect in my career and personal life, and for your dedicated Dutch translation of the Samenvattig of my thesis. Silvia, thank you for your company and support throughout my journey. We share happiness and sorrow, and grow stronger together. My sincere gratitude also goes to my teachers and classmates at the Gerrit Rietveld Academy. It takes great courage to develop my Alter Ego as a visual artist besides being a scientific researcher; thank you all for keeping me going this far and staying true to myself. Thanks to all those who dropped a random “how’s going” that distracted me from a blue, unproductive day.

I cannot express how grateful I am to my beloved family. Thank you, Fanyu, my husband and best friend in life, for your love, encouragement, patience, tolerance, and understandings, for your constructive and practical suggestions as a mature PhD researcher, and for respecting and supporting my bold decisions and wildest dreams. Thank you, my mom Yuehua Zhang, for your unconditional love and for being my role model as a decent, modest, strong, and independent woman. I’m always proud of being your daughter and I hope at this moment you feel proud too. To Amélie, Heidi, Penny, Pavarotti, Yellow, Grey, and Pipi, the cutest little living creatures keeping my company through the lonely days.

I’m so lucky and proud to have you all along my journey.

Bo Zhang Amersfoort, The Netherlands, January 2021

iii

Content

1. Introduction ... 1

1.1General introduction and problem statement ... 1

1.2Development of automated driving ... 2

1.3Levels of driving automation and human factors issues ... 4

1.4Control transitions in driving automation ... 9

1.5Research objectives and research questions ... 14

1.6Thesis structure ... 16

References ... 18

2. Determinants of take-over time from automated driving ... 27

2.1Introduction ... 28 2.2Methods ... 30 2.3Results ... 36 2.4Discussion ... 46 2.5Supplementary material ... 50 Acknowledgement ... 50 References ... 50

3. Transitions to manual control from highly automated driving in non-critical truck platooning scenarios (with hand-held NDT) ... 65

3.1Introduction ... 66

3.2Methods ... 69

3.3Results ... 73

3.4Discussion ... 78

Acknowledgement ... 81

Reference ... 81

4. Transitions to manual control from highly automated driving in non-critical truck platooning scenarios (with mounted NDT) ... 87

4.1Introduction ... 88 4.2Methods ... 88 4.3Results ... 91 4.4Discussion ... 96 4.5Conclusion ... 97 Acknowledgement ... 98 References ... 98

5. Taking back manual control after automated platooning: A comparison between car and truck driver’s behaviour ... 101

5.1Introduction ... 102 5.2Methods ... 106 5.3Results ... 111 5.4Discussion ... 118 5.5Conclusion ... 120 Acknowledgement ... 121 References ... 121

6. The effect of see-through truck on driver monitoring patterns and responses to critical events in truck platooning ... 129

6.1Introduction ... 130

6.2Method ... 131

6.3Results and Discussion ... 133

6.4Conclusion and Outlook ... 137

Acknowledgement ... 137

References ... 137

7. The effects of monitoring requests on driver attention, take-over performance, and acceptance ... 139 7.1Introduction ... 140 7.2Methods ... 144 7.3Results ... 152 7.4Discussion ... 162 7.5Conclusion ... 165 Acknowledgements ... 165 References ... 165

8. Conclusions, discussion and implication for practice ... 171

v

References ... 179

Summary ... 181

Samenvatting in het Nederlands ... 185

About the author ... 189

vii

List of Abbreviations and Acronyms

ACC Adaptive Cruise Control

ADAS Advanced Driver Assistance System

AD Automated Driving

ADS Automated Driving System

AOI Area of Interest

BE Brake Event

BRT Brake Response Time

CACC Cooperative Adaptive Cruise Control

CAV Connected and Automated Vehicle

DDT Dynamic Driving task

DOF Degree of Freedom

HMI Human-Machine Interface

ICC Interclass Correlation Coefficient

ITS Intelligent Transportation Systems

LKAS Lane Keeping Assist System

MD Manual Driving

MRT Movement Response Time

NDT Non-Driving Task

ODD Operational Design Domain

OEDR Object and Event Detection and Response

PRT Perception Response Time

RT Response Time

SAE Society of Automotive Engineers

SD Standard Deviation

SDLP Standard Deviation Lane Position

SPD Speed

STS See-Through System

SWA Steering Wheel Angle

THW Time Headway

TO Take-over

TOT Take-over Time

TRT Total Response Time

1

1. Introduction

1.1 General introduction and problem statement

Motor vehicles radically changed the way we live and work, ever since its introduction 130 years ago. Although they have often been a symbol of independence and freedom (Steg, 2005), times are changing and the negative impacts are getting more and more attention, such as road accidents, traffic congestion, and pollutant emissions. According to the World Health Organization (2018), every year more than 1.3 million people worldwide are killed, and up to 50 million are injured in road traffic crashes. The National Highway Traffic Safety Administration (NHTSA) estimates that “dangerous choices or errors people made behind the wheel” contribute to more than 90% of serious crashes, which can be caused by various human-related factors such as intentional violations, attention lapses, distraction, fatigue, and alcohol use. Automated driving technology has great potential to fundamentally solve road traffic issues and improve our quality of life (Fragnant & Kockelman, 2015; Kyriakidis et al., 2019). By assisting or replacing human operation in normal and critical driving tasks, serious crashes that are dominantly caused by human errors are assumed to be largely reduced. If connected and coordinated through advanced communication systems in the form of a platoon, automated vehicles are able to travel safely even with much smaller headways, which can largely increase road capacity, improve traffic flow efficiency, and reduce energy consumption (Coppola & Silvestri, 2019; Rios-Torres & Malikopoulos; 2017, Shladover, 2018; Talebpour & Mahmassani, 2016).

Decades of effort and technology advancement seem to bring automated vehicles from science fiction fantasy closer to reality, but a long and windy road is still ahead to the realization of full automation where no human intervention would be needed in any road situation. Safe human-system interactions, particularly at transitions of control to the driver when the human-system cannot cope with the current driving situation, pose key challenges for a successful deployment of automated intelligent systems at different stages of development. When the system is less capable and reliable, the driver has to closely monitor the system and take over imminent manual control when necessary. This challenges humans’ inherent weak point of staying vigilant over a prolonged period of time (Mackworth, 1948; Davies & Parasuraman, 1982;

Parasuraman 1987), and drivers’ capability to respond adequately within a short time budget (Banks, Eriksson, O'Donoghue, & Stanton, 2018; Casner, Hutchins, & Norman, 2016). When the technology becomes more mature, the system is supposed to largely replace human operation in most situations. However, even in these cases, driver interventions due to system limitations or failures, or the exceedance of the system’s operational limits are still needed. A main challenge at this stage is how drivers respond in these conditions and how they can be supported in taking back control in a safe and smooth manner.

Since the past decade, an increasing number of studies have addressed human factors issues related to control transitions (for an overview, see Lu, Happee, Cabrall, Kyriakidis, & De Winter, 2016 and Kyriakidis et al., 2019), and suggest that multiple factors related to the driver, the automation system, and the situation potentially influence driver readiness to take back control. This means that no single criterion for an optimal take-over time budget exists today that fits all drivers in all situations, and design solutions are called for to support individual drivers in taking over control. To achieve this goal, a good understanding of driver take-over process and the variability between and within drivers is needed, which requires further efforts because the large majority of studies merely focus on mean take-over response times measured in stand-alone automated car scenarios. This thesis tackles the issues stated above and investigates driver behaviour and performance at control transitions and the variability, particularly in automated platooning scenarios. The aim is to contribute to designing safe and comfortable control transitions to manual.

The following part of this chapter first provides a state of the art on development of automated driving (Section 1.2), then introduces levels of driving automation and related human factors challenges (Section 1.3), followed by fundamental knowledge on transitions of control, driver take-over process, and driver take-over performance (Section 1.4). In Section 1.5, the research objectives and research questions are formulated and explained. The overall structure of the thesis is outlined in Section 1.6.

1.2 Development of automated driving

Even long before the wish to solve traffic safety issues by means of automated vehicles, people started to dream of cars driving by themselves. One very early prototype of “driverless” cars dates back to mid-1920s when Houdina Radio Control demonstrated “American Wonder” in New York – a 1926 Chandler controlled by the following car via radio impulses (Time Magazine, 1925). At the 1939 World’s Fair in New York, General Motors sponsored the “Futurama” exhibit to envision American lifestyle 20 years in the future. The highlight was an infrastructure system that could guide radio-controlled cars through electromagnetic fields embedded within the roadways, which was generally seen as the first proposal of an automated highway system in the world. In the late 1950s, General Motor together with the Radio Corporation of America (RCA) brought this idea to life and demonstrated a full-size electric guide-wire system on a test track that enabled automated lateral and longitudinal control of vehicles. Research and development that revolved around this concept continued for another 20-30 years in the United States, UK, Germany, and Japan.

From the 1980s, the rapid advances in electronics, computers, communications, controls, and sensor technologies started to shape modern automated driving systems that operate mainly based on onboard sensors and control units without dedicated infrastructure support (also known as stand-alone automation systems). In the early 1980s, Ernst Dickmanns and his team at Bundeswehr University Munich developed the first automated vehicle of this type: a Mercedes van incorporating a vision-based system that was capable of detecting road markings

and controlling steering wheel, throttle, and brakes of the van based on real-time evaluation of image sequence. In successive years, various projects tackling technological challenges in automated driving and practical road traffic problems were launched worldwide, including the EUREKA Prometheus project of the European Union (Williams and Preston, 1987), the DARPA Autonomous Land Vehicle (ALV) project in the United States (Schefter, 1985), and the Super-Smart Vehicle Systems program in Japan (Tsugawa, 1991). These intensive research and development efforts largely increased the capability and efficiency of vehicle automation, as evidenced by a series of successful demonstrations and challenges conducted during the 1990s and 2000s, such as DARPA Grand Challenges (Buehler, Iagnemma, & Singh, 2007) and the Urban Challenge (Buehler, Iagnemma, & Singh, 2009).

In the late 2000s and the early 2010s, Google and many major automotive manufacturers initiated commercial research on automated driving systems and began various testing on public roads. Meanwhile, several core advanced driver assistance systems (ADAS) such as adaptive cruise control (ACC) and lane keeping assist systems (LKAS) were gradually introduced to the market. Driving automation began to receive considerable attention in mass media, raising increasing interest in publicity and industry. In 2015, Tesla became one of the first car manufacturers to release partial automated driving features (Autopilot) to its customers, followed by other major automakers including BMW, Mercedes Benz, Audi, and VOLVO. Incorporating multiple advanced sensors (e.g., stereo camera, radar, and ultrasonic sensor) and enhanced processing capabilities, these commercialized automation systems are able to conduct longitudinal and lateral control of the vehicle in simple traffic situations under non-adverse weather conditions. Despite the image that is being presented by some users, industry or the media, the driver of these commercially available vehicles has to constantly monitor the driving environment and be prepared to take immediate control when necessary since its functioning is not reliable yet. Around 2015, at the time of the introduction of Autopilot by Tesla, the concept of automated driving reached its peak in expectation in the Hype Cycle (Figure 1.1 Left), implying an expected mainstream adoption within 5 to 10 years

As more time passed, the high expectations have diminished to a more realistic level in 2019 (Figure 1.1 Right), due to increasing real-world experience with automated driving technology and a better understanding of its capabilities and limitations (for an overview of expert opinions, see Bazilinskyy, Kyriakidis, Dodou, & De Winter, 2019). For example, it is now more widely recognized that currently, commercially available automation systems requiring constant driver supervision involve safety risks (for an overview of expert opinions, see Kyriakidis et al., 2019), as evidenced by a number of fatalities involving Autopilot that occurred in recent years (Mider, 2019). Also in the Netherlands, the Dutch Safety Board has identified a number of new road safety risks associated with current commercially available vehicles with automated functions (Dutch Safety Board, 2019).

Figure 1.1: Gartner Hype Cycle for Emerging Technologies illustrated for the year 2015 (Left) and 2019 (Right). The red circles highlight the position of automated driving technology.

In parallel with the development of stand-alone automated vehicles, extensive research on connected automated vehicles began in the late 1980s with the California PATH (Partners for Advanced Transportation Technology) program. The core concept is to operate vehicles in platoons, in which virtually connected vehicles travel closely together as one cooperative system, with its primary goal to maximize highway capacity, energy efficiency, and safety (Shladover, 2006). The early research of PATH focused on passenger cars platoons, revolving around the idea that all vehicles (including the platoon leader) would be fully automated on dedicated lanes to eliminate negative impact caused by human error. Since the late 1990s, research and development interest has shifted towards platooning of heavy-duty trucks, largely driven by the fuel economy in freight transportation (Tsugawa, 2013). Advanced longitudinal control functionalities that combine onboard sensors and vehicle-to-vehicle (V2V) communication, such as cooperative adaptive cruise control (CACC), have also become subjects of intensive research to achieve more flexible and reliable platooning systems. Although platooning has not yet been deployed in commercial use, current efforts are made towards operating truck platooning in real life cases and implementation of multi-brand platooning (e.g., ENSEMBLE, see Willemsen et al., 2018). A milestone is the European Truck Platooning Challenge initiated by the Dutch EU Presidency, in which six European truck manufactures brought truck platoons onto public roads for the first time, travelling from various European cities to the final destination of the port of Rotterdam in the Netherlands in April 2016. Due to safety concern and legal issues, the trucks participating in the challenge only performed automated longitudinal control, despite the capability of automated steering as demonstrated on test tracks, leaving an important role for the human in the platooning scenarios.

1.3 Levels of driving automation and human factors issues

It can be seen that automated vehicles of various forms are merging into our roadways more or less along an evolutionary path. As the capability of a driving automation system increases, a wider range of driving tasks could be carried out by the system, leading to a reduction in human engagement and a change in the role of the human driver. It is highly important to provide common classifications that facilitate the exchange of knowledge across domains, and to avoid confusion and imprecisions when describing system functionalities and limitations (Shladover, 2018).

Michon’s (1985) hierarchical structure is typically referenced to categorize manual driving tasks, which comprises three levels. The lowest level (operational level) concerns longitudinal and lateral motion control to maintain the vehicle’s lane position in traffic, which are normally carried out with little cognitive effort. At the intermediate level (tactical level), driving manoeuvres are planned and executed based on the pre-defined goals and in response to the objects and events in the driving environment, which normally requires greater mental efforts, and more elaborate physical movement. Examples of such manoeuvres are a lane change, obstacle avoidance, and overtaking. At the highest level (strategical level), the general planning of a trip is conducted, including scheduling of the trip and selection of destinations and routes. At the current stage, automated vehicles are mainly expected to carry over driving tasks at the operational and the tactical levels, which are often referred to as dynamic driving tasks (SAE, 2018; Merat et al., 2019).

The German Federal Highway Research Institute (BASt; Gasser & Westhoff, 2012), the United States National Highway Traffic Safety Administration (NHTSA, 2013), and the Society of Automotive Engineers (SAE, 2018) have each developed a taxonomy of levels of driving automation. Despite differences in definitions and terminologies, the three taxonomies share similar criteria to categorize the automated driving systems, mainly based on how primary dynamic driving tasks (i.e., longitudinal and lateral vehicle motion control, and monitoring of the driving environment) are distributed between the human driver and the automation system (Lu et al., 2016). Because the SAE taxonomy provides relatively more precise definitions and is the most-cited reference for automated-vehicle capabilities in both industry and academic research (Shuttleworth, 2019), it is adopted in this thesis and is explained below (according to the latest version released in 2018).

As depicted in Table 1.1, six levels of driving automation are defined by SAE (2018), ranging from SAE L0 (no driving automation) to SAE L5 (full driving automation). Differences between levels are primarily determined by means of who is performing the dynamic driving task, who is the fallback-ready agent (i.e., who performs the dynamic driving tasks in case of system failures), the limit of the operational design domain (ODD, the specific conditions under which the system is supposed to function), and the role that is required of the driver in that specific level.

SAE L0 is equivalent to manual driving, which means that the driver executes all dynamic driving tasks, possibly assisted by lower levels of ADAS that provide warnings or momentary assistance (e.g., emergency braking assistance).

In SAE L1 (Driver assistance), the driving automation system executes either longitudinal or lateral control of the vehicle, while the driver performs the remaining dynamic driving tasks. Examples of such systems are the Adaptive Cruise Control (ACC) or Lane-Keeping System (LKS).

In SAE L2 (Partial driving automation), vehicle motion control in both dimensions are executed by the system (e.g., combining ACC and LKS), but the driver is required to constantly monitor the driving environment and supervise the system, and intervene as necessary to maintain safe operation of the vehicle even without notification. From this level on (L3 and higher), the human driver’s role as an active operator is fundamentally changed.

In SAE L3 (Conditional driving automation) and L4 (High driving automation), the system performs all dynamic driving tasks within the limit of the ODD, so the driver does not need to permanently monitoring the driving environment and is allowed to engage in non-driving tasks. This is the level where automated driving becomes interesting for users, since they can make

better use of travel time for work and relaxation. A SAE L3 system expects the driver to intervene in case the system approaches the limits of its ODD (for instance when approaching a workzone or extreme weather conditions) or when system failures occur (i.e., the driver serves as the fallback-ready user within a reasonable time budget because the camera is obstructed). The system is capable to determine the necessity for driver intervention and issues a timely request to intervene (also known as take over request (TOR) in a rich body of literature). In SAE L4, the system does not primarily rely on the driver to be the fallback-ready user, even though a request to take back control may be provided. In case a system failure occurs or the driver does not respond to the issued request, the system performs the fallback itself and transitions automatically to a minimal risk condition (e.g., conducting an emergency stop at a ‘safe’ spot). Currently, SAE L4 automation is predominantly developed for public transport concepts such as last mile transits and automated shuttle buses on limited trajectories. Public transport solutions will not be part of this thesis since there is no transition of control back to manual driving.

In SAE L5 (Full driving automation), the system is capable of all dynamic driving tasks in all situations (i.e., the ODD is unlimited) without involvement from human drivers. However even there, some situations may occur where the car cannot continue, such as flooded roads or extreme snow storms. Since L5 vehicles are not designed for a transition of control, this level will also not be discussed in this thesis.

There have been discussions that the taxonomies with numbered levels may induce misinterpretation and false expectation among the public, because the ascending levels do not necessarily correspond to the actual evolution of the technology (Templeton, 2014). For instance, a L3 system may be only available during traffic congestions with speeds under 50km/h in the operational design domain, whereas on a trip without any congestion, it will not have this level available at all. And even with a L4 system, it still may be the case that it may only work on motorways for 10% of the time, and not be available on other roads or under adverse weather conditions. Some also argue that the SAE levels are not sufficient to describe the variety of automation systems, for instance public transport on pre-defined routes only, or systems that are related to connected and cooperative technology (e.g., automated platooning systems). Due to the very short inter-vehicular gaps, it is very risky for drivers in a platoon to respond to longitudinal critical events in case of system failures, so the platooning system should perform the fallback even in lower SAE levels. In addition, the lead vehicle is normally operated by a professional driver in a lower level of automation (e.g., in SAE L0 or L1) than the following vehicles (e.g., in SAE L3 or L4), which makes it difficult to apply the SAE taxonomy to the whole platooning system.

Despite the limitations of the SAE levels to describe all possible types of automation, the key element remains the role of the human driver in driving tasks. Irrespective of whether a vehicle does or does not use cooperative technology or has an extended or a limited operational design domain, it is important to know if the driver needs to monitor the road or not, and what will happen when a driver needs to intervene under various conditions. While it may be true that only fully automated vehicles that completely remove driver responsibilities throughout the ride could maximize safety benefits of driving automation, experts generally believe that there is still a long way to go until this ultimate goal can be achieved (Kyriakidis et al., 2019; Shladover, 2016; Gomes, 2014; Underwood, 2014; Yoshida, 2014). As long as human intervention in any form is still expected, a safe human-automation interaction would play a central role in a successful deployment of driving automation on public roads (Carsten & Martens, 2019).

Table 1.1: Summary of levels of driving automation defined by SAE (2016). DDT = Dynamic driving task, OEDR = Object and event detection and response (incl. monitoring the driving environment and the automation system), ODD = Operational design domain, ADS = Automated driving system.

An increasing number of studies have addressed human factors issues that hinder drivers’ capabilities to maintain safe control at different levels of automation (Saffarian, De Winter, &

Lev

el

Name Narrative definition

DDT DDT fallback ODD Sustained lateral and longitudinal vehicle motion control OEDR

Driver performs part or all of the DDT 0 No Driving

Automation

The performance by the driver of the entire DDT, even when enhanced by active safety systems.

Driver Driver Driver n/a

1 Driver Assistance

The sustained and ODD-specific execution by a driving automation system of either the lateral or the

longitudinal vehicle motion

control subtask of the DDT (but not both simultaneously) with the

expectation that the driver

performs the remainder of the DDT.

Driver and system

Driver Driver Limited

2

Partial Driving Automation

The sustained and ODD-specific execution by a driving automation system of both the lateral and

longitudinal vehicle motion

control subtasks of the DDT with the expectation that the driver completes the OEDR subtask and supervises the driving automation system.

System Driver Driver Limited

ADS (“System”) performs the entire DDT (while engaged)

3

Conditional Driving Automation

The sustained and ODD-specific performance by an ADS of the entire DDT with the expectation that the DDT fallback-ready user

is receptive to ADS-issued

requests to intervene, as well as to

DDT performance-relevant

system failures in other vehicle

systems, and will respond

appropriately.

System System Fallback-ready user (becomes the driver during fallback) Limited 4 High Driving Automation

The sustained and ODD-specific performance by an ADS of the entire DDT and DDT fallback without any expectation that a user will respond to a request to intervene.

System System System Limited

5

Full Driving Automation

The sustained and unconditional

(i.e., not ODD-specific)

performance by an ADS of the entire DDT and DDT fallback without any expectation that a user will respond to a request to intervene.

Happee, 2012; Van den Beukel; & Martens, 2013; Cunningham, & Regan, 2015; Kyriakidis et al., 2019; Navarro, 2019). In partial automation (SAE L2), the current state of technology requires the driver to be prepared for imminent intervention and does not allow drivers to be able to take their eyes off the road. Therefore, it is crucial to constantly keep the driver in the monitoring loop, so that a high level of situation awareness (Endsley, 1995) can be maintained. The problem with this is that humans are by nature poor at vigilance tasks (i.e., to sustain concentrated attention and respond to irregular and infrequent target stimuli for extended period), as is the case with monitoring driving automation systems (Onnasch, Wickens, Li, & Manzey, 2014; Norman, 2015). Accumulating evidence suggests that vigilance performance (e.g., detection accuracy and response speed) inevitably declines over time on task due to the depletion of attentional resources (Mackworth, 1948; Davies & Parasuraman, 1982; Parasuraman 1987; Scerbo, 2001). The monotonous nature of monitoring tasks would also induce intentional or unintentional attention switching towards task-unrelated thoughts and stimuli (Scerbo, 1998; Helton et al., 2005; Casner & Schooler, 2015), and consequently cause a loss of situation awareness.

In addition, overreliance (or complacency), resulting from inappropriately high trust (overtrust) in automation, is another major cause for monitoring failures (Ensley, 2017; Singh, Molloy& Parasuraman, 1993; Parasuraman & Riley, 1997; Carsten & Martens, 2019). Poor trust calibration is directly associated with an inaccurate mental model of system capabilities and limitations, which can be caused by insufficient information provided about system functionalities, little or no feedback on the system status, and a lack of prior experience with such systems (Lee & See, 2014; Endsley, 2017; Walker, Wang, Martens, & Verwey, 2018; Carsten & Martens, 2019). Also, for people it may seem counter-intuitive that they are driving cars with automated functions, without any of the benefits of vehicle automation such as being able to (temporarily) do something else.

In higher levels of automation (SAE L3 and L4), the driver would be allowed to be temporarily out of the monitoring loop and to engage in a wide range of non-driving tasks (Naujoks, Befelein, Wiedemann, & Neukum, 2017), which would result in a large variability in drivers’ activities, mental states, and body postures. How the systems can adapt to broad variations within and between drivers in taking over control, becomes the main challenge at this stage. Great caution is also needed to minimize automation surprises caused by unexpected system performance (Sarter & Woods, 1997), which are more prone to occur with increasing system reliabilities (Carsten & Martens, 2019; Endsley, 2017). Although ideally the system should only be allowed to be activated under conditions that it can cope with, accidents cannot always be avoided. Another potential issue induced by extensive use of automation is the loss of manual control skills, which has been frequently found among pilots that have become accustomed to autopilot systems (Veillette, 1995; Young, Fanjoy, & Suckow, 2006; Haslbeck & Zhang, 2017). Safety critical situations would occur when the driver has to take over control due to system failures, while he/she is no longer proficient in manual driving.

The human factors challenges described above largely revolve around transitions of control from automation to the driver and vice versa. A good understanding of drivers’ performance at control transitions in various states and conditions remains substantial for the development of safe driving automation platforms. In the following section, we introduce the mechanism of control transitions in driving automation, driver take-over process, and the measures for driver take-over performance.

1.4 Control transitions in driving automation

Transitions of driving tasks between the human driver and the automation system may occur at different automation levels, due to various reasons such as drivers’ personal preferences, entering or exiting the ODD, sensor limits or in extreme cases malfunctioning This can be further differentiated between transitions of control and transitions of monitoring activity (Lu et al., 2016). A transition of control mainly involves a reallocation of vehicle motion control tasks, while a monitoring transition concerns a change in status between driver (temporarily) monitoring and system (temporarily) monitoring. An overview of all possible transitions is illustrated in Figure 1.2.

Figure 1.2: All possible transitions between the driver and the automation system at different levels of automation, adapted from Flemisch, Kelsch, Löper, Schieben, & Schindler, (2008) and Lu et al., (2016). The solid green lines indicate transitions of both longitudinal and lateral vehicle motion control; the dashed green lines indicate transitions of vehicle motion control in only one dimension; the orange lines indicate monitoring transitions.

Martens et al., (2008) outlined three fundamental questions to classify control transitions: 1) who has it (who conducts the control task at the start of the transition); 2) who should get it (who should conduct the control task after the transition); and 3) who initiates the transition. Based on this classification scheme, Lu et al., (2016) further integrated the underlying reason of the transition into their framework, (i.e., whether the transition is mandatory based on certain decision rules or requirements, or optional based on the driver’s voluntary intention while both agents are capable of the control task), yielding six types of control transitions between the driver and the automation as depicted in Figure 1.3.

Control transitions to the automation are normally performed within the ODD of the system. In optional Driver-Initiated transitions, the driver decides to activate the automation system because he/she feels like doing so, such as switch on the ACC on a normal highway. In turn, the driver may choose to deactivate the automation and start driving manually even though the system is functioning well. Mandatory Driver-Initiated transitions concern situations where the driver has to switch on the automation to avoid or minimize undesirable consequences or safely-critical situations. For example, the driver has to activate the automation in order to enter an area specially equipped for automated vehicles, or to join in an automated platoon. Another possible use case is that the driver hands over control to the automation because he/she is no longer able to drive safely (e.g. due to emergency health issues or with driver fatigue). Mandatory transitions to automation could be automatically initiated by the automation system as well, depending on the implemented transition strategies. Currently, these transitions do not exist yet in current levels of automation of commercially available vehicles.

Mandatory control transitions to the driver (also referred to as driver take-over) are generally seen as a major challenge facing automotive engineers and human factors researchers, which would occur upon exceeding the system’s ODD or due to system failures. In such situations, the driver is forced to take over control because the automation can no longer drive safely or cannot drive safely under conditions that are coming up soon. Safety critical situations would occur if the driver cannot intervene adequately in time.

In Automation-Initiated transitions, the system is able to determine the necessity to transfer control to the driver and issue a TOR, which allows the driver to respond within a certain time budget before causing adverse consequences (e.g., a collision) or triggering the system fallback performance. The available time budget may vary largely between different types of take-over scenarios (Gold, Naujoks, Radlmayr, Bellem, & Jarosch, 2018). If, when or where the system reaches its functional limit can be estimated in advance from system backend, map or V2X communication (i.e., scheduled take-over). Therefore, the system is able to provide a timely TOR with a longer time budget. The take-over scenario is normally more critical when it is related to the behaviours of other road users or system failures (i.e., unscheduled take-over). The available time budget depends on the predictability of the unfolding situation and the capabilities of the onboard sensors.

In Driver-Initiated transitions, the driver diagnoses that the system can no longer handle the current situation based on kinematic feedback from the vehicle, cues in the driving environment and his/her expectations, and takes over control without being requested by the system. This type of transition would occur when the system is not acting according to what a driver expects, or when a system is not able to detect (in time) a critical event or fails to diagnose its malfunction (i.e., silent automation failures, Louw, Kuo, Romano, Radhakrishnan, Lenné, & Merat, 2019), which is more likely in lower levels of driving automation. The hazardous situation could have already become highly critical when it is detected by the driver.

1.4.1 Driver take-over process and take-over performance

As with most of the human executed activities in dynamic environments, taking over control in response to external input involves multiple psychological and motor processes. Referencing Wickens’ information processing model (Wickens, Hollands, Banbury, & Parasuraman, 2015), the take-over process covers 1) the detection and perception of the take-over stimulus (i.e., a TOR or an environmental event that can initiate a driver take-over), 2) cognitive processing of the stimulus (to comprehend the necessity to take over control) and the current driving situation (to determine how to take over control), 3) establishment of motor readiness by repositioning

the hands on the wheel and foot on the pedal, and 4) execution of an action that influences vehicle motion control (by steering, pressing the braking/gas pedal, or pressing a button to disengage the automation).

Wickens et al. (2015) described speed, accuracy, and attentional demand as “the big three” measures of human performance. Generally speaking, the faster, the more accurately, and the more effortlessly a task is being conducted, the better the performance. Correspondingly, driver take-over performance can be evaluated using measures related to the response time (RT) to complete the over process, the quality of the manoeuvre that is required for a specific take-over scenario, as well as the workload involved in the process. They are introduced in turn below.

Take-over time

Driver take-over response time, or in short take-over time (TOT), is an essential parameter to evaluate driver take-over performance. A successful take-over first requires the driver to respond before the situation exceeds his/her controllability (Nilsson, Falcone, & Vinter, 2015). TOT is generally measured from the onset of the take-over stimulus (a TOR or an environmental event) until the driver makes a conscious intervention (Gold & Bengler, 2014).

Additional response time metrics can be measured for a sequence of actions to break down the take-over process, in order to analyse driver behaviour at a fine-grained level. An overview of measurable RTs is given in Figure 1.4. In a few studies, gaze reaction time, eyes-on-road time, and hands-on wheel time were registered to reflect the moments when the driver senses the take-over stimulus, starts perceptual and cognitive processing of the driving environment, and establishes motor readiness, respectively (e.g., Gold, Damböck, Lorenz, & Bengler, 2013; Körber, Gold, Lechner, & Bengler, 2016; Feldhütter, Gold, Schneider, & Bengler, 2017). Less commonly registered is the initial start of the driver’s hand movement (Kerschbaum, Omozik, Wagner, Levin, Hermsdörfer, & Bengler, 2017; Kerschbaum, Lorenz, & Bengler, 2015), possibly because time consuming video annotation is needed. RTs measured until the first hand movement provide some insight in the time elapsed until the driver has comprehended the necessity to take over control and starts to regain motor readiness. This is similar to perception time in the concept perception-response time to analyse drivers’ braking response in the manual driving context (Olson, 1986; Green, 2000). Movement time, or motor response time (corresponding to the second component of perception-response time), can be measured from the start of the movement until the moment when the driver grasps the steering wheel, which reflects the time it takes to execute the actual response to establish motor readiness.

It has to be noted that the RTs above may only apply to the analysis of more basic responses towards the take-over stimulus. How long it takes for the driver to fully comprehend the different aspects of the driving situation and how the process of selecting a specific driving manoeuvre is difficult to observe and determine. These processes may overlap with other information processing stages and may even continue after the start of the manoeuvre. The driver may respond inadequately or even incorrectly if he/she takes over control before acquiring a sufficient level of situation awareness, particularly in complex and critical situations.

Figure 1.4．Measurable response times (RT) within the take-over process in temporal order, adapted from Gold et al., (2013) with modifications.

Take-over quality

Besides take-over time, the quality of the take-over performance is also important to be measured. The evaluation of take-over quality depends on required actions to handle a certain take-over scenario (Gold et al., 2018; International Organization for Standardization [ISO], 2020). In some simple scenarios, such as taking over at the end of an ODD zone or due to the absence of lane markings, operational actions to stabilise the vehicle in its lane is sufficient. Desired performance is to maintain a steady lane position, a smooth, appropriate speed, and a safe following distance to the front vehicle. Correspondingly, metrics related to the variability of lane position (e.g., Standard Deviation Lane Position, SDLP), steering activity (e.g., the number of steering wheel reversals per minute), driving speed, and time headway can be employed to assess the quality of lateral and longitudinal driving performance. The performance data are often assessed in small windows after the control transition to explore the course of manual driving performance “recovery” as a function of time or travelling distance (Merat, Jamson, Lai, Daly & Carsten, 2014; Skottke, Debus, Wang, & Huestegge, 2014; Pfromm, Khan, Oppelt, Abendroth, & Brudera, 2015; Eriksson & Stanton, 2017). In order to estimate when the carryover effects of automated driving have sufficiently diminished and the driver could continue driving in a safe manner, driving performance after a take-over is often compared to driving performance in a baseline manual driving condition or reference threshold (ISO, 2020). More complex take-over scenarios require the driver to perform tactical manoeuvres according to pre-defined goals or rules, such as lane changing and turning, stopping at a traffic light, and adjusting speed to a new speed limit. These types of scenarios are less addressed in empirical studies. Whether the driver was able to respond correctly and timely to achieve the specific goal is the main criterion used in studies for the assessment of take-over quality.

In most challenging scenarios, the driver has to perform imminent tactical actions to achieve the goal of avoiding a collision with obstacles or other road users. Quality assessment for collision avoidance scenarios mainly concerns if the driver can operate the vehicle in a relatively safe manner and as a minimum can prevent a collision without endangering him or herself and other road users. First it has to be evaluated if crashes or other non-controllable events occur (skidding, rotating, swerving across several lanes or off paved road), which self-evidently would indicate take-over failures (Naujoks, Wiedemanna, Schömig, Jarosch, & Gold, 2018). For successful take-overs, low risk should be involved in the manoeuvre, which can be

estimated using surrogate safety metrics such as minimum time to collision (TTC) and minimum clearance towards the obstacle or other road users (Gettman & Head, 2003; Happee, Gold, Radlmayr, Hergeth, & Bengler, 2017; Tarko, 2018). A higher value indicates a larger time gap or spatial safety distance and a lower degree of endangerment (Naujoks et al., 2018). Measures related to acceleration profiles such as maximum longitudinal and lateral acceleration are commonly used to reflect the intensity of the executed manoeuvres. Lower accelerations generally suggest smoother and safer manoeuvring. Both TOT and take-over quality should be taken into consideration to determine take-over performance. While a faster response is generally preferable, it can be assumed that drivers are more prone to making errors. Such a speed-accuracy trade-off (Fitts, 1966; Wickelgren, 1977) in control transitions has been suggested in a number of empirical studies that reported a higher take-over quality when drivers responded later (e.g., Damböck, Weissgerber, Kienle, & Bengler, 2013; Gold et al, 2013; Clark & Feng, 2017; Ito, Takata, & Oosawa, 2016). To find a “sweet spot”, balancing take-over quality and TOT is a key aspect in determining a desirable take-over time budget and providing implications for the development of driving automation systems.

Workload

Workload can be understood as the amount of work or effort necessary to perform a task, which concerns both physical and mental aspects (Meijman & Mulder, 1998). When taking over control, the driver experiences physical workload to reposition his/her hands back on the wheel and feet back on the pedals, and mental workload to process the take-over stimulus and the take-over situation, and to make decisions on a take-over action. The physical load would be relatively small, while the mental workload may vary largely depending on task difficulty, situation complexity, and the driver’s capability and states (Lee, Regan, & Horrey, 2020). There are three main categories of workload measurement techniques: subjective measures, physiological measures, and performance measures (De Waard, 1996; O’Donnell & Eggemeier, 1986). Subjective measures assess and quantify personal judgements of experienced workload, which are usually well-established rating scales, such as the NASA-TLX (Task Load Index) (Hart & Staveland, 1988), the SWAT (Subjective Assessment Technique) (Reid & Nygren, 1988), and the RSME (Rating Scale Mental Effort) (Zijlstra & Van Doorn,1985). Physiological measures are used to infer variations in workload through changes in an individual’s physiological states. Commonly used measures concern brain activity (EEG), cardiac activity (heart rate, heart rate variability, blood pressure), respiratory activity (respiration rate), eye activity (pupil diameter, eye fixation), and galvanic skin response (De Waard, 1996; Charles & Nixon, 2019). Performance-based measurement techniques are developed based on the assumption that the human operator has limited attentional resources (Kahneman, 1973; Yeh & Wickens, 1988). Decrements in primary task (i.e., the task of interest) performance can be an indicator that the workload is too high or too low, as both overload and underload can diminish performance. The performance on an additional, low-priority task, also called secondary task, can reflect the remaining capacity of the operator while performing the primary task (see e.g., Jahn, Oehme, Krems, & Gelau, 2005; Martens & Winsum, 2001; and Verwey, 2000 for applications of secondary task measures in the manual driving context).

In the context of automated driving, driver workload during the use of automation systems is a frequently recurring research topic (e.g., Stanton, Young, & McCaulder, 1997; Heikoop, De Winter, Van Arem, & Stanton, 2019; Stapel, Mullakkal-Babu, & Happee, 2019; see De Winter, Happee, Martens, & Stanton, 2014 for a review), while workload directly related to control transitions is hardly touched upon. One possible reason is a lack of suitable workload measures. Physiological and performance measures are usually aggregated over time and consequently

not applicable for assessing workload in a fragment of seconds (De Waard, 1996; Verwey & Veltman, 1996). They are also likely to interfere with the take-over process involving rapid physical movements. Subjective assessment causes least intrusion, but may induce biases if the driver cannot precisely recall workload experienced during one specific control transition, especially when multiples transitions are performed within one drive. Focusing on safety aspects of control transitions, this thesis will mainly assess time and quality aspects of take-over performance, which are direct indicators of transition safety and can be objectively assessed in simple, and non-intrusive manners.

1.5 Research objectives and research questions

This thesis focusses on Human Factors issues related to the transition of control from SAE L2-L4 vehicle automation to manual control, in both stand-alone automated driving scenarios and automated platooning scenarios. The research in this thesis addresses the challenges of take-over times and take over quality in various conditions. In order to contribute to a better understanding of behaviour during transitions of control to manual driving, the following research objectives are proposed:

• Obj. 1: Explore determinants of TOT and TOT variability in normal and critical

take-over scenarios and gain a deeper insight in the actual driver take-over process.

The first objective is to study the factors that affect TOT under various circumstances, in order to get more insight in inter- and intra-individual differences. Knowing the factors that affect TOT may help support drivers in taking back control in a safe and smooth manner. Although existing research provides useful insight into some factors that affect TOTs, findings of the individual studies are hardly generalizable across different driving contexts, because only a small number of variables are manipulated per study. Up to now, little effort has been made to quantitatively synthesize all the available TOT studies for a more holistic picture. In addition, most studies only focus on mean or median TOT values without addressing intra- and inter-individual differences (Dinparast, Djadid, Lee, Domeyer, Schwarz, Brown, & Gunaratne, 2019; Eriksson & Stanton, 2017; Mole et al., 2020). Nevertheless, outliers in the response time distribution are most prone to safety critical accidents and other adverse situations (Horrey & Wickens, 2007). This would indicate that taking mean TOT of one study as the basis for understanding how long it takes before a driver takes back control may lead to an unsafe design. More research efforts are needed to focus on variability in TOTs in various conditions, and to investigate the cause for large outliers. Wickens & Corlett (2015) even claim that the ultimate goal is to design safe automation systems that can accommodate as close to 100% of the target driver population.

• Obj. 2: Study driver take-over time and performance with professional drivers in

automated truck platooning scenarios.

The second objective is to fill in the research gap of TOT in platooning scenarios, particularly that of professional truck drivers. Truck platooning is considered as a first step towards automated freight transportation in an open and uncontrolled environment (Bhoopalam, Agatz, & Zuidwijk, 2018; Janssen, Zwijnenberg, Blankers, & De Kruijff, 2015; World Maritime University, 2019). Nevertheless, the large majority of human factors studies merely focus on passenger car scenarios in a non-platooning situation. Research on professional truck drivers’

take-over behaviour is very limited, and at the start of this thesis, we were not aware of any attempts that were made to systematically study driver take-over performance when leaving an automated platoon. It still remains to be understood if the specific features of platooning, such as the very short inter-vehicular distance and blocked front view, and the driver categories (professional or non-professional drivers), influence the way drivers take over control. If we are able to identify the specific requirements of platoon drivers, and the differences between professional truck drivers and normal passenger car drivers in take-over behaviour, we can deliver valuable input for designing safe control transitions in platooning systems.

• Obj. 3: Explore potential approaches that prime drivers for a safe and smooth

take-over.

The third objective is to design and evaluate solutions that prime drivers for a safe and smooth take-over. As mentioned in the previous sections, a TOR with a large time budget cannot always be provided, particularly in on-road settings where critical yet unpredictable situations may occur. However, it is ineffective and unrealistic to require the driver to sustain attention and stay prepared to take back control throughout the ride. It is important to explore possible countermeasures that adapt to the uncertainty and complexity of the road situations and allow the driver to allocate attention accordingly. As mentioned above (Obj. 1), several previous studies pointed to a large variation between individual drivers in taking over control, and no single take-over time budget exists that fits all situations. The feasibility of an adaptive and personalized control transition approach will also be discussed in this thesis.

Based on the research objectives, the following research questions are formulated:

• RQ1: What factors influence driver response times in taking back control from

automated to manual driving?

This research question focuses on TOT and mainly links to research Obj. 1. To answer this question, a comprehensive literature search and meta-analysis has been conducted to provide the state of the art on driver take-over research, and to explore determinants of TOT on an aggregated level (i.e., what determined the mean/median TOTs). Data collected from empirical driving simulator studies performed in this thesis provide additional insight into influencing factors that were not included in the meta-analysis.

• RQ2: How do car drivers and professional truck drivers perform when decoupling

from highly automated platoons in normal, non-critical situations under the influence of various task conditions?

This research question focuses on driver behaviour at control transitions in platooning scenarios, and mainly links to research objective 2. This question is answered by analysing car and truck drivers’ performance data when taking over in car and truck platooning driving simulator studies.

• RQ3: Could a monitoring request help driver respond more adequately with

take-over performance in critical take-take-over situations?

This research question focuses on designing and evaluating an innovative HMI concept that may prepare the driver for potential critical takeovers, which mainly links to research Obj. 3. To answer this question, an empirical car driving simulator study is conducted that compares drivers’ take-over performance and subjective ratings when using the innovative system to the conventional system that only issues TORs.

• RQ4: What explains variability in driver take-over times and is an adaptive

approach tuned to a specific driver or conditions a feasible solution for a safe and smooth transition to manual driving?

This RQ focuses on variability in TOT between and within drivers (Obj. 1), based on which the feasibility of adaptive driving automation tuned to a specific driver’s states is discussed (Obj. 3). This question is answered combining the meta-review study, from which the correlation between the mean and standard deviation of TOT can be yielded, and all empirical studies performed in this thesis, which allow inspection into individual drivers’ takeover response.

This research contributes to the literature on Human Factors in transport, primarily in the domain of highly automated driving in the context of driver behaviour at transitions of control. Especially, this research is one of the initial studies that focuses on automated truck platooning and adaptive automotive automation. The results contribute to the development of an improved human-system interaction for comfortable and safe transitions of control.

1.6 Thesis structure

This thesis consists of eight chapters. The structure of the thesis and the links between the chapters and the research questions are depicted in Figure 1.5, and explained below.

In Chapter 1, the general research background, research objectives and research questions have been introduced.

Chapter 2 presents an exhaustive meta-review of 129 driver take-over time studies. The aim of this meta-review is to provide the state of the art on the relevant research on TOT, and to explore the effects of a wide range of factors on driver TOT and its variability on an aggregated level. This study mainly aims to answer RQ 1. Three complementary meta-analytical approaches were employed: (1) a within-study analysis, in which differences in mean TOTs were assessed for pairs of experimental conditions, (2) a between-study analysis, in which correlations between experimental conditions and mean TOTs were assessed, and (3) a linear mixed-effects model combining between study and within-study effects.

Chapter 3, 4, 5 and 6 present empirical studies that systematically investigate driver take-over performance in platooning scenarios. Data for driver performance analyses were generated from three simulator-based studies conducted in the project Adaptive Virtue Tow-Bar (A-VTB) within TNO’s Early Research Program Human Enhancement. Chapter 3 and 4 describe two truck platooning studies with professional truck drivers in non-critical scenarios. In Chapter 5, we compared a passenger car platooning study with one truck platooning study to explore the difference between car and truck drivers. Chapter 6 concerns truck drivers’ take-over performance in a critical system failure scenario.

The four chapters together provide a holistic picture of drivers’ take-over performance in platooning scenarios and address RQ 2. In addition, we conducted manual video annotations to break down the total TOT. TOT was divided into the perception response time and the hand movement response, in order to analyse driver take-over processes at a fine-grained level. Video recordings of participants’ behaviours during the take-over process were also analysed to explore individual differences. This contributes to a better understanding of the driver take-over process and contributes to RQ 1. A more detailed overview of each chapter is presented below.