The application of Virtual Reality Technology for PCIT

The application of Virtual Reality Technology for PCIT

SUBMITTED IN PARTIAL FULLFILLMENT FOR THE DEGREE OF MASTER

OF SCIENCE

Paul Schrijver

6376975 / 10116052

MASTER

INFORMATION

STUDIES

G

S

FACULTY OF SCIENCE

UNIVERSITY OF AMSTERDAM

July 18, 2018

The application of Virtual Reality Technology for

PCIT

Paul Schrijver University of Amsterdam

Thesis project Information Studies - Game Studies Paul.Schrijver@student.uva.nl

ABSTRACT

Parent-Child Interaction Therapy (PCIT) is a form of be-havior therapy that treats children that show Disruptive Behavioral Disorders, by, among others, training their par-ents. In the first phase of PCIT parents often struggle with learning to use child-directed interaction skills. This research aims to find out whether Virtual Reality (VR) can be applied for providing complementary training in this first phase of therapy since VR has shown promising results in the field of behavior therapy and for use in education and training. A prototype application was made using A-Frame that fo-cuses on training three interaction skills. The prototype was evaluated by five field experts. The results look promising, VR seems to be a good fit for the therapy and the prototype could be tested in clinical practice after minor revisions. KEYWORDS

Virtual Reality, PCIT, training, 360-degree video

1 INTRODUCTION

Over the years Virtual Reality (VR) has become a more common phenomenon. Especially the use of Virtual Reality through Head-Mounted Displays (HMDs) has seen a surge in recent years, thanks to affordable hardware, such as the Samsung Gear VR line of products [13, 16]. Because of this both the amount of VR related research and real-world ap-plications of VR technology have seen in increase in a very diverse fields of applications [18, 44]. The unique proper-ties of VR, such as providing users with an immersive and realistic experience [45], make it interesting for ‘serious’ applications such as in health care.

Especially within the context of mental health care VR has become very popular. This popularity is specifically no-ticeable within behaviour therapy [4, 24, 46] and exposure therapy [7, 33, 36, 38, 42]. The advancements in VR tech-nology and growing amount of applications have also been noticed by researchers at ‘De Bascule’, an academic centre for youth psychiatry that works together with the Academic Medical Center in Amsterdam.

De Bascule offers a wide range of treatments and thera-pies, one of them being Parent-Child Interaction Therapy (PCIT) which is focused on children with behavioral issues. All children experience losing their temper and becoming frustrated. It is completely normal for children to show ag-gression [2]. However, when this sort of behavior becomes common and persistent they could develop into Disruptive Behaviour Disorders (DBDs). PCIT focuses on helping chil-dren overcome these issues. The therapy applies to chilchil-dren that are approximately in the age range of 3- to 6 years-old. The therapy targets both child and parent and attempts to improve interaction through multiple phases of therapy [2].

The first phase of PCIT is very much based on improv-ing communication between parent and child by increasimprov-ing parental responsiveness and establishing a secure and nurtur-ing relationship. This is done by teachnurtur-ing the parent several child-directed interaction skills, which for example can be labeled praises or reflections. These skills are taught to par-ents by a therapist coaching them during ‘special playtime’ [50]. This takes place at De Bascule, a child and its parent are placed in a special play room with a one-way mirror. Behind the mirror is a therapist observing the playtime whilst coach-ing the parent by givcoach-ing instructions that are transferred by the ear-piece a parent is wearing.

Therapists at De Bascule describe that during the therapy sessions parents tend to respond well to the coaching and the sessions have positive result. However, parents have to practice their skills at home as well. This is done by organiz-ing their own special playtime sessions with their child. In this setting it turns out parents often experience difficulties since they are not being coached and can be thrown off by an often busy environment. The difficulties that are experienced mostly consist of when to use a skill, and what to say exactly. This is where the question arises whether a complementary form of training can be developed that focuses on these ob-stacles. This form of training should be usable by parents at home without supervision of a therapist.

An application that recreates the special playtime sessions that include coaching is deemed to be useful additional train-ing for parents. Considertrain-ing the before mentioned use of VR and results from studies in related fields of application, VR seems to be a promising technology to apply in this specific problem area. Currently no research is published on the ap-plication of VR, or any form of multimedia technology for that matter, for PCIT. Therefore this study investigates, in co-operation with De Bascule, whether VR could be applica-ble to create a complimentary tool to support the first phase of PCIT. This leads to following research question:

Research question Can Virtual Reality technology be applicable for providing complementary training in the first phase of Parent-Child Interaction Therapy? In order to research this a literature review is conducted to explore both the theory behind the technology and providing context on related work. Following that a prototype has been developed. In order to accurately replicate the special playtime experience, the virtual world that is presented has been made with VR video, also called 360-degree video. The resulting prototype is evaluated by field experts, such as PCIT therapists and researchers. Based upon their analysis the viability of the concept is determined.

2 LITERATURE REVIEW

Virtual reality technology

Virtual Reality has been well defined by Brooks: "A virtual reality experience is an experience that immerses the user in a responsive virtual world. This implies user dynamic control of viewpoint.[12]. So in essence Virtual Reality revolves around immersing a user in a different, virtual, environment. Essential for this immersion is giving a certain amount of control within the virtual world that is being conveyed [12]. This comes down to (at least) offering users the possibility to control their viewpoint by moving their heads for example. There are many different technologies developed for con-veying a virtual world. Examples of these are panoromic displays, cave automatic virtual environments (CAVE) and Head-Mounted Displays (HMDs) [12]. In recent years HMDs got more popular and have seen a lot of development. This is not without reason, as HMDs can provide the high level of immersion that is sought after for VR [37]. Compared to other systems HMDs are also practical due to their size. Developments in HMDs also resulted into high quality VR experiences becoming more accessible and affordable for large audiences [27]. Furthermore HMDs have been shown to provide a particularly high level of immersion, adding to the VR experience [21, 25, 40].

VR and HMD technology does come with challenges how-ever. One of these challenges is the phenomenon simulator sickness. The symptoms of simulator sickness are compa-rable to those of motion sickness and they can occur when a user is immersed in a virtual environment. Some of the symptoms are eye strain, dizziness and nausea. This leads to an unpleasant experience that also breaks immersion [39]. This is caused by movements in the virtual environment not being synchronized or lining up to movements of the head. In order to minimize the risk of simulator sickness occurring, system latency should be kept at a minimum. The amount of latency in a system correlates to the amount of risk for simulator sickness to occur [15]. Another key factor is frame rate per second (FPS). With higher frame rate simulator sick-ness is less likely to occur [26]. There are no exact numbers for this, but some software developers suggest a minimum frame rate as high as 75 fps [43].

VR technology offers the possibility to emerge somebody in a different, virtual world which can have benefits with regards to, among others, training and medical applications. Using HMD’s offers most immersion. For a VR experience to be effective and pleasant, simulator sickness has to be prevented as much as possible.

Virtual Reality for learning and training

Already in the early nineties of the last century, first ex-ploratory studies were published investigating the added value of virtual reality learning environments [9]. It was found that VR learning environments can provide powerful learning experiences that might be more powerful than tra-ditional learning experiences. The first-hand experience that virtual worlds can offer provide a completely different world of learning with the potential to be more effective [9]. Tech-nical challenges were mostly obstructing VR applications at the time, but technical progress in the years after resulted in real-world applications. In 1999 Brooks made an overview of the state of VR [12]. His findings showed successful Virtual Reality applications in, among others, the field of training pi-lots and surgeons. Despite these successful examples, Brooks still described VR as ‘barely working’. This was caused by the still present technical difficulties and limitations, such as severe system latency and quality of displays. [12]

Further progress in technology in the following decade resulted in more and more applications of VR for learning purposes. Since the mid 2000’s many examples can be found of Virtual Reality for learning, such as applications for teach-ing history, storytellteach-ing and sports simulations. Reflectteach-ing on these applications they show promising results. The Vir-tual Learning Environments prove to be flexible in terms

of application and meet the common demands of education [8, 34, 47]. Furthermore they turn out to be effective as teach-ing material and prove a useful innovation to ‘old-fashioned’ education [34]. More recent research shows similar positive results [30]. Teaching through virtual reality can be more effective than traditional methods. This is both with respect to learning and knowledge retention in both training and learning situations [30, 41]. Applications still need to have a solid design from a pedagogic point-of-view however, and should not rely merely on the positive effect of VR [19].

In the first phase of PCIT, teaching parents interaction skills and training them how to put them into practice plays a main role. VR has shown promising results in the field of learning and training. The technology can offer a unique learning effect by means of immersion, but application design should not merely rely on these effects.

Virtual Reality in psychology

Within the field of health care there are many different ap-plications of VR. The field of psychology however matches with the context of this research. Currently there are no VR applications, or other digital applications, used in the field of PCIT. In the field of psychology VR is already being applied however.

One example of is the use of VR for behaviour therapy. Be-haviour therapy is a form of therapy that aims to treat many forms of mental disorders by helping the patient change cer-tain behavior [46]. PCIT is an intervention that is also a form of behaviour therapy. It has become popular to use VR for behavioral therapy, although the maturity of the technology is still questioned. The reason why it is popular is because of the potential it has shown in research so far. Clinical in-terventions are highly controllable when they are based on VR, negating random elements that may occur in real-world environments [24]. These advantages can also apply to the first phase of PCIT. For example, when parents are practicing special playtime at home, random factors could compromise the session.

A form of behavioral therapy where VR has been applied is public-speaking anxiety therapy [4]. This is one of many examples. Again the great amount of control over the envi-ronment was a motivation to use VR for this purpose. It has shown to be effective, lowering patients’ anxiety over the course of the therapy. This is comparable to regular therapy, but with great practical benefits. The study also suggests using VR for treating other phobias [4]. Both regarding be-havioral therapy and exposure therapy, this is exactly where VR is already being applied ever since. Exposure therapy aims to treat anxiety disorders by exposing the patient to the

source of anxiety. Virtual Reality Exposure Therapy (VRET) has made its way to the clinical practice [28]. VRET has shown to be effective in many cases [7, 33, 36, 38, 42]. The ef-fectiveness of VRET is very positive and similar to behavioral therapy that incorporates VR [33]. Just as with behavioral therapy, the amount of control is a great advantage over reg-ular exposure therapy. Next to the high degree of control, VR is found to be able to provide enough realism and immersion for the therapy to be effective. Other advantages are high cost effectiveness, flexibility and it can be used for patients that do not dare to face the ‘real’ version of their source of anxiety [33, 36]. Limitations are mostly of technical nature, not all anxieties can be easily replicated in a virtual world and some technology was not progressed far enough at the time some research was conducted [36]. The latter is less common in more recent research.

The use of VR in the field of psychology, it is even already being used at De Bascule for treating anxiety disorders, pro-vide a solid base of understanding for using VR for PCIT. Advantages such as giving a high amount of control and negating external, distracting factors pave the way for using VR for PCIT.

360-degree video

There are different technologies for creating virtual environ-ments. One of these is creating a world using 3D technology. This is often a time consuming method with relatively high complexity, requiring the maker to have 3D modelling skills [23]. Because of this, it does not seem like a good solution for creating a prototype. A virtual environment can also be created using 360-degree video, also called VR video. This is relatively far less complex and more cost-effective and there-fore seems like an appealing option. In turn, this technology does have to offer sufficient immersion to be effective as a virtual world.

Recent research has shown that 360-degree video can of-fer a higher degree of immersion than regular video. When the footage is converted to a stereoscopic image, which can be displayed using a HMD, a completely immersive 3-dimensional experience can be created [22, 49]. There is no evidence using VR video is this manner lacks immersion or realism compared to other technologies. It does however limit movement to three degrees of freedom. This means the virtual environment can only be explored by head movement, so by looking around [6]. It is not possible to move through the environment, which is a feature in other technologies that offer six degrees of freedom.

However VR video is a powerful platform that has been shown to, for example, help understand a wide variety of

complex human interactions, such as instructional discourse [35]. Research using 360-degree video in training, such as more efficiently learning practical skills [49], and teaching context have shown promising results. In some contexts it increased students’ learning [5] and it proved a useful tool in training presentation skills [48]. This, combined with a high curiosity motivation [20], can make it an attractive tool for training purposes. Therefore it seems like a suitable option in the context of PCIT.

With regards to simulator sickness 360 degree video also has advantages. As described, this form of sickness is mostly caused by movements in the virtual world not aligning with movements in real-life. Since VR video only offers three de-grees of freedom, the amount of movement is limited. It is suggested that symptoms of simulator sickness when us-ing 360-degree video are mostly caused by camera move-ments [20]. Using a stationary camera is an easy way to work around this [20]. Simulator sickness can still occur, but one of the factors causing it can be limited.

3 METHODOLOGY

In cooperation with several therapists at De Bascule a con-cept for the VR application was formed. Together with the therapists it was decided the application would have to focus on difficulties parents experience in the first phase of PCIT. In the ideation phase first a concept was drafted where a parent would have to make choices in a virtual conversation with a child, choosing what they would say in a given sit-uation. Based upon that choice the virtual interaction with the child would continue, thus also displaying the result of wrong decisions. Later this concept was deemed not to be suitable, because showing what not to do is likely not to have the desired learning outcome. Parents could get confused or negatively stimulated. Because of this the concept was further refined. Instead of wanting to also show wrong de-cisions, it was decided to only show what to do in a certain situation.

The final concept consisted of a VR application that fo-cuses on training three Child-Directed Interaction Skills that are being applied during the so called special playtime: la-beled praises, reflections and behavior descriptions. These are described as the ‘Do’ skills, with a positive following [50]. These specific skills were chosen because they are found to be difficult to master. Despite parents understanding what these skills contain, they have difficulties determining what to say when. The application would offer the parent the choice of which skill to practice. Upon choosing the desired skill, a virtual world is displayed that represents special playtime. A child is shown in a fitting setting, after which the play session starts. This takes approximately five minutes, just as

its non-virtual counterpart. The child starts playing but the virtual world is ‘frozen’ as soon as the chosen skill can be put into practice. This gives the parent time to think about what to say in the given context. After giving the parent some time to think, a phrase is displayed suggesting what would have been a correct phrase to say. The parent can reflect on this, after which the play session continues. This was deemed an effective way of teaching the correct moment to use a certain skill, complemented by a fitting suggestion. Based on the literature review, the most suitable form of VR for the desired goal would be the use of HMDs. The level of immersion and realism provided allows for the accurate representation of special playtime that is sought after in this project. Accessibility to HMD technology does form a con-straint however. It has to be taken into account that parents should be able to use the training application at their own homes. Therefore high-end HMDs that are run by powerful computers are not an option. Instead it was opted to make use of smartphone-based HMDs. The availability of smart-phones among patients is deemed high. These devices can be combined with universal HMDs such as Google Cardboard viewers.

Creating the prototype

A prototype needed to be developed that resembles the spe-cial playtime experience as closely as possible. This needed to be done in such a way that the prototype would be afford-able, has low complexity and is as universally applicable as possible regarding devices that are available to the target audience.

It was decided creating a virtual world using 360-degree video would be both fitting and feasible. The benefits of this technology do not outweigh the relatively limited movement in the virtual world since that is not a crucial part in special playtime. Since no suitable software was available to display the content on a potentially wide variety of mobile devices, custom software had to be developed.

Software.To successfully display 360-degree footage using an HMD by means of a smartphone, some core functions are required. First of all the footage has to be split into two separate images, one for each lens in the HMD. Each image needs a slight horizontal offset of the viewport to create an illusion of depth. To counter the distorting effect caused by the lenses of the HMD, barrel distortion has to be applied to the image as well. This is displayed in figure 2. With these measures, a stereoscopic image pair can be created, poten-tially resulting in a reasonably high-fidelity VR experience. To be able to let a user move the image in accordance to head movement, different motion sensors equipped to the smartphone have to be accessed.

Table 1: Specifications of Motorola G5S Plus Specifications

Display size 5,5" Display resolution 1920 x 1080

SoC Qualcomm Snapdragon 625

CPU Octa-core Cortex-A53 @ 2.0GHz

GPU Adreno 506

OS Android 7.1.1

Browser Mozilla Firefox

To make the software both as universally applicable and cost-effective as possible, it was decided to make use of the Google Cardboard VR platform. This platform provides a standardized way of implementing all functionality men-tioned before [29]. An added benefit of this ‘standardized’ platform is that it allows third-party manufacturers to pro-duce HMDs that comply to it. This results in a wide variety of available HMDs that are available for prices as low as €5,-. Furthermore this type of HMD has been shown to provide a sufficiently immersive and effective VR experience even when compared to high-end HMDs such as the Oculus Rift [3].

The following requirements were formulated for the pro-totype:

• Playback of high-quality 360 degree footage • Conform to the Google Cardboard standard • Display a simple and user-friendly user interface • Limited footprint on the mobile device, due to memory

and storage limitations

• Lightweight application to ensure compatibility with lower-end smartphones

To investigate how these requirements would best be put into practice, two different test versions of the prototype were made. One version was implemented in the game engine Unity1_{, the other was made using A-Frame}2_{. The findings}

of both versions were compared to choose the most suitable option for this case.

Unity.Unity 2017.3.0f3 was used to create the first test ver-sion of the prototype. For video playback the integrated video player component was used. Video was projected within a sphere to incorporate the 360-degree aspect of the footage. The Google VR SDK for Unity3_{was used to include Google}

Cardboard4 _{compatibility. To reduce the footprint of the}

1_{https://unity3d.com/} 2_{https://aframe.io/}

3_{https://developers.google.com/vr/develop/unity/get-started-android} 4_{https://vr.google.com/cardboard/developers/}

application, all video content was streamed from a remote server. Using the Android SDK5_{, the application was exported}

to Android and tested on a Motorola G5S Plus smartphone6_,

running Android 7.1.1. This smartphone is modern, but the hardware specifications are relatively on the lower end when it comes to processing power and graphical performance. However, the specifications still allow it to run 4K video at 30fps, which the minimum that is being kept for this proto-type. This is done to ensure a sharp, high resolution image that does not need additional anti-aliasing post-processing. The specifications of this phone are shown in table 1.

A-Frame.A-Frame is an open-source, JavaScript based, web framework developed by Mozilla for creating web-based VR experiences. The framework allows developers to imple-ment ‘WebVR’ applications in a relatively simple fashion [1]. A test version of the same application as was made with Unity was created using A-frame. In turn this was tested with the aforementioned Motorola phone, but also with an iPhone 7 running iOS 11.3.

Comparison.After creating the test versions, they were compared based on costs, overall performance, compatibility, possibilities with regards to future features, flexibility and ease of use, both in terms of development and usability. The main findings of comparing the applications are shown in ta-ble 2. Since the Unity application runs as a native application on the mobile devices, there are some advantages consider-ing hardware support. The engine can efficiently make use of the available computing power. Unity as a mobile engine has proven itself over the years and is very popular. However, native applications have to be distributed through app stores, such as the Google Play Store and the Apple App Store. This often requires licensing fees, which are also in question for Unity itself when using the engine professionally. The inte-grated video player in Unity does allow for external content to be displayed, but will download that content completely upon run time. Considering the generally large size of 360-degree media, this results in long loading times. Integrating media in the application itself drastically increases the size of the application, which is not preferable due to limited storage capacity on mobile devices. The overhead of the Unity appli-cation was already rather large, being over 20 Megabytes in size.

The A-Frame application was fully web-based, negating the need for a native application except for the web browser that is used to display it. Furthermore, the built-in video

5_{https://docs.unity3d.com/Manual/android-sdksetup.html}

6

Table 2: Comparsion of A-Frame and Unity

A-Frame Unity

Advantages Disadvantages Advantages Disadvantages

Low overhead Requires a recent device,_{OS and browser} Well-documented No streaming video_{with default video player} Supports streaming

video HEVC dependanton browser Proven technology License fees Supports many

VR platforms WebVR is stillexperimental Hardware support Necessity to distributethroug (paid) app stores Supports a wide

variety of OS’s High overhead (∼20MB)

Lacks codec support player supports real-time streaming, resulting in low load

times. With an overhead of less than 1 Megabyte, the appli-cation was much smaller than its Unity counterpart. Because the application is web-based, it can be run on a wide variety of devices, as long as the browser has built-in WebVR sup-port. All modern mobile web browsers support this. Initial tests regarding video playback with the Android device were excellent, however the iPhone showed severe frame drops. Despite the Apple device being far more powerful than the used Android device, it seemed the (experimental) WebVR implementation on iOS lacked optimization. Repeating the tests a few weeks later showed similar results to the previous Android test. This shows how experimental this technology still is.

Because of the size of 360-degree video, it is necessary to compress the footage. It would be preferable to use High Efficiency Video Coding (HEVC, also known as H.265), due to its high compression rate, whilst maintaining quality. The Unity video player does not support video that is encoded using HEVC. A-Frame however uses the HTML5 video player that is embedded in the browser used to run the application. Because most web browers for Android do not support HEVC, it was opted to instead use H.264.

Because of the higher flexibility, low footprint, lack of any (licensing) fees and good test results, it was decided A-Frame was the better choice for the final prototype. Since both A-Frame and WebVR are both in active development, it seems likely most of the disadvantages will be resolved in the near future.

Creating the training footage

Goal of the footage is to provide an authentic experience that comes as close as possible to the five minutes of special playtime parents have to practice at home. In order to do so the PCIT manual was closely followed to correctly recreate a special playtime setting [17]. This setting was validated

Figure 1: An impression of the set.

by a PCIT therapist. Three different clips were shot, one for each skill that can be practiced.

The recordings were made with a four year-old child as an actor. The child did not have any disruptive behavior disorders, nor was she involved in any form of PCIT or other therapy. The ‘set’ consisted of a living room where the child would either sit on the ground or at a table, using toys that can be played with independently and don’t lead to any wild games or unwanted behavior. In practice, a set of train tracks, building blocks and a colouring book were used. The latter was recorded while the child was sitting at a table, the rest of the games were played sitting on the ground. Figure 1 gives an impression of the setting. The child was alone in the living room, her parents and the researcher were in the adjacent kitchen, just outside of her view. Communication was still possible though, which was necessary to give the child instruction without being in view of the 360-degree camera.

The camera that was used was a Garmin VIRB 360. Cam-eras that can record 360-degree footage are still relatively in their infancy, especially regarding affordable consumer cameras. This consumer camera however seemed the best option due to its image quality, which is important to provide

an immersive experience, overall maturity of the device and affordability. The camera was set up to record in ‘4K’, result-ing in an equirectangular image with a resolution of 3840 x 2160 at 30 frames per second. The device was mounted on a custom-made tripod that was set at approximately eye height of where the parent would sit in the given situation. Two days were drawn out to have as many recording sessions as possible, without causing strain on the child. In the end, around one and a half hours of footage was shot, which had to be reduced to three clips of five minutes. For each child-direct interaction skill a clip was created. Editing the footage consisted of making corrections to the image quality (improvements of lighting and colour display), audio (mostly removing unwanted noises such as prompts to get the child to say something), adding the ‘training content’ and encoding the clip in H.264 at a bitrate of around 10 Mb/s. In each clip there are ten moments to use a certain skill. This amount is in accordance to the end goal of the first phase of PCIT [17]. At each of these moments the video is frozen. Each time the video freezes, five seconds of thinking time are provided so the user can think of a phrase. These five seconds are in compliance to the PCIT manual [17] that suggests using a reflection within that time span after a child has said something. This interval was also used for the other skills. After the thinking time passes, a phrase is displayed that would be appropriate to use in that situation. This is displayed for four seconds, providing enough time to read the phrase and reflect upon it. Figure 3 shows a phrase being displayed. The clip used to train reflections was provided with subtitles, because it was fairly hard to exactly understand what the child was saying due to an accent.

All phrases that were suggested in the clips were either literally copied examples from the PCIT manual [17], or based on them. The coding scheme with priority order pro-vided in the Dyadic Parent-Child Interaction Coding System (DPICS) manual was used to properly code all phrases. As a final check, the phrases were validated by one of the PCIT therapists at De Bascule.

Resulting prototype

After creating the three clips, they were integrated in the previously made software. A simple main menu was added, consisting of the three buttons labeled with the skills avail-able for practice. This can be seen in figure 2. A round cursor is displayed in the center of the image. Upon hovering over a button, this cursor changes size, to inform the user interac-tion with the button is possible. Selecting a button is done by pressing a button on the HMD. After selecting a skill, the corresponding clip starts playing. After the clip ends, a menu

Figure 2: Split image with barrel distortion, showing main menu.

Figure 3: The application suggesting a phrase after the think-ing time.

is displayed giving the user the option to either re-watch the clip, or return to the main menu.

4 EVALUATION

In order to evaluate the prototype it was important to know if the application is usable, especially for the specific tar-get audience, and if it provides a fit with PCIT. To gather information on this five field experts, varying from PCIT therapists to DPICS experts, where asked to help evaluate the application. The amount of five experts was chosen be-cause this would be considered sufficient expertise of PCIT and it has been shown that 85 percent of usability problems tend to show with this sample size [32].

Procedure

The evaluations were done in a one-on-one setting that was recorded in both audio and video. This was to prevent dis-tractions caused by taking minutes and keep the attention of both researcher and participant. At first the participant was given some contextual information about the project and the basics of the application were explained, without going into much detail. The application was run on the same Motorola

G5S Plus as used in prior testing. To prevent any problems with wireless internet connection the entire application was run locally from the Android file system instead of from a remote web server.

It was demonstrated to the participant how the application was started and how the phone was mounted in the HMD, a Homido V2. After setting up the HMD, it was explained how the HMD is operated and the participants were asked to take a seat in a rotating desk chair. Subsequently the participant was asked to put on the HMD and make sure it was both comfortable and snug.

The participants were asked to think aloud from this point on, in order to gather data on usability and other impressions [31]. They were given the task to start the training video that focuses on reflections and watch it completely. The video was chosen because it provides a very clear example of the concept and it does not leave a lot of room for interpretation with regards to what can be said in a certain situation. This room for interpretation could be a distraction, where experts focus more on the exact contents instead of the application itself.

After completing the training video they were asked to start another video, which they could choose themselves. After about a minute of looking at the successive video, the participant was asked to remove the HMD. The second video was cut short because its main purpose was to force the par-ticipant to navigate through the menus and get an impression of a video with a slightly different setting. The video focused on reflection shows the child sitting at a table, whereas the other two videos show the child playing on the ground. This gives a slightly different effect.

Thereafter the participants were asked to fill in the System Usability Scale, being a ten item questionnaire with a five-point Likert-scale. This questionnaire was chosen because it provides a quick and reliable measurement of usability of the application. It can be used on wide variety of systems such as the given prototype [10]. A score can be calculated based on the answers, which gives an impression of the degree of usability. The standard template for SUS was used [11], meaning it was in English. To not alter the validity of the test it was not translated. After the first two evaluations, the word ‘cumbersome’ in question eight turned out to be difficult to translate for the participants, so a translation was later added. The questionnaire is added in appendix A.

Final part of the procedure was a semi-structured inter-view. This was done to further elaborate on how participants experienced the prototype, next to what they already said

during the demonstration itself, and discuss the practical application and relation to PCIT. The following questions were asked in a comparable capacity, in an order that fit the specific conversation:

• Do you feel the application is suitable and usable for the target audience?

• Do you feel the videos are a realistic representation of special playtime?

• Do you feel the application fits the first phase of PCIT? • Do you feel the application suits the problems

experi-enced by parents in the first phase of PCIT? Analysis

All gathered data was analyzed after the evaluation sessions were finished by means of the recordings. To analyze the results of SUS, every questionnaire was scored based upon the SUS manual [11]. The final outcome of the SUS test is the average of the five scores.

Everything that was said by participants during the think-aloud was summarized and noted. The same applies for ob-servations that were made during the demonstration, which gave an impression of how participants were interacting with the software.

To analyze the interviews a thematic analysis was con-ducted. Themes were based on recurring subjects and the ‘fixed’ questions from the interviews [14]. The resulting themes were ‘suitability for parents’, ‘connection to PCIT’, ’connection to PCIT problems’, ‘realism’, ‘usability’ and ‘dizzi-ness’. Per interviewee and theme a short summary was added to a spreadsheet. The findings resulting from the demonstra-tion were also added, often providing more context to certain remarks. The resulting spreadsheet gave a complete and con-cise overview of the session in a thematic structure.

5 RESULTS

A few recurring remarks stood out from the evaluation ses-sions immediately. All of the participants described the expe-rience as being very fun and valuable. They were initially ‘in awe’ when looking through the virtual world. To prevent a positive bias resulting from this, during the interviews there was plenty of space created for more critical analysis.

One of the main themes was usability. In order to be able to add value to PCIT the prototype had to be usable. The resulting average score from the SUS questionnaire was 88.5 with a standard deviation of 7.8. The scoring table is added in appendix B. A score of 68 is considered average [10], showing this score is well above average. This indicated the system scores very well with regard to usability and was easy to use.

This corresponds to observations that were made during the evaluations, all participants navigated through the applica-tion with ease and confidence. It was also often emphasized by participants that the system was very simple and looked appealing. A couple of recurring remarks concerning usabil-ity were the volume of what the child said and readabilusabil-ity of the text. To make it easier to follow what the child says, the volume should be increased or headphones should be used. The positioning of the text could be better, by placing it closer towards where the participant is looking. It would also be easier if some of the longer lines were displayed for a longer period of time. Participants were asked if they experience any dizziness (simulator sickness). One of the participants did mention dizziness was starting to occur at the end of the demonstration. Another participant experienced dizziness from the beginning, which developed into slight nausea at the end of the demonstration. Both these demonstrations took about seven minutes. Other participants did not experi-ence any dizziness, nausea nor had other complaints.

Participants broadly agreed about whether the application is suitable for the target audience. They thought it is suitable for parents that are involved in PCIT and also simple enough to be usable. It is thought the application will appeal to the imagination and provide a fun exercise. However there were some critical remarks. First of all the cost of an HMD could be a barrier if parents have to buy this themselves. The audience is very varied, and not every parent is as involved with the training. The experts speculated that this specific group might either not use this application at all, because it might feel like just another chore. However it is also speculated that because this an accessible, fun and easy way of practicing, they might actually be motivated to use it. The application might be even more suitable for a larger group of parents if another child of a different gender is added, in order to make the videos more in accordance to their own situation. The fact that the application is complementary to the therapy makes that parents can choose whether they would like to use it, and when. This lowers the threshold to try it and might create a positive, low-stress learning environment because parents can choose a quiet moment to get extra practice. Especially in the first weeks, this extra practice can be very valuable since a lot of new skills and information are conveyed in this period.

The application is seen as a very good fit to the first phase of PCIT. The chosen interaction skills are seen as very im-portant to the therapy and the way they are trained closely resembles the first coaching sessions of PCIT. Although par-ents will receive the same information that the videos convey in the shape of a paper manual, the videos are deemed to be more likely to be used. The possibility was mentioned

of expanding the application with training videos of other skills that are taught in the first phase of PCIT. Other than this, no remarks were made on how the prototype fits in with the first phase of PCIT.

In addition, the application was deemed to address prob-lems parents experience in the first phase of PCIT. The appli-cation was found especially helpful with regards to parents developing creativity on what to say to their children. Most experts also agreed the software helps with learning to rec-ognize moments to use a specific skill. However one of the experts argued this aspect needs extra work, because in case of the reflection training video, for example, there where some moments unused where a reflection could have been used. All in all the application was deemed to be effective in helping parents overcome some of the issues they are experiencing in the first phase of PCIT.

Before the participants were even asked about realism or immersion, they all mentioned something related to this. Most remarks were very similar, a selection of comments made by the participants: "It really feels like I’m there, like there’s a child sitting in front of me.", "This is so funny, it’s like you’re in a different world and the child is sitting there with you." This showed how the application seemed to reach its goal of being an immersive experience. Multiple remarks where made about how the camera standpoint was properly chosen to give an impression of presence. Two of the experts did mention how it was preferred to have a slightly wider field of view to give a broader view.

All experts agreed the videos resemble a real-life example of five minutes of special playtime. With more interaction between child and viewer, this would have been even better, for example if the child would look in the camera while speaking. Also the behaviour of the child in the video was marked as exemplary. This was not seen as a problem, since it can even be seen as the end-goal of the therapy. However it does not necessarily represent behaviour of the children that are involved in the therapy, especially in the first weeks. It was suggested that adding content where a child shows more challenging behaviour might be interesting.

6 DISCUSSION

The results reflect positively on using the prototype in clin-ical practice. However, before putting the application into practice, some improvements can be made to the prototype. These improvements are insights gained from further testing and the expert evaluations. It’s recommended to increase the volume of the child to make it easier to understand. This could also be solved by making the user wear headphones, but increasing the volume would negate this as being a ne-cessity. The prompt texts can be made more user friendly by

moving them closer to where the user is looking, the child. Instead of placing them above the child they might be better placed below the child. In addition, providing a couple of seconds more reading time would make it easier to read the texts. Instead of just relying on the user to read the texts, its advisable to narrate the texts by means of voice over. This would also make the application more accessible, especially for people with dyslexia for example. Currently only sub-titles are added to the video that is focused on reflections, adding subtitles to the other videos would make the child easier to understand. Finally, a slight increase of the field-of-view might improve the overall experience and lower the chances of users experiencing dizziness. This should be com-bined with using a camera that can record at frame rates over 30FPS in order to rule out simulator sickness as much as possible.

Next to conducting research that puts this prototype into practice to measure its effectiveness, it would also be interest-ing to compare this prototype to another prototype that does not use VR. It would be possible to make a similar application that reuses the same footage and displays it on a regular, flat (smartphone) screen. This way a comparison can be made to the VR version to see if the theoretical benefits are actually measurable in terms of effectiveness. Considering the extra complexity and costs of VR it would be recommended to confirm its added value.

The videos themselves were found to be already suitable training videos. Future videos can be improved by adding some more interaction and the child showing more variety in behaviour. It was found that doing the recordings with a four year-old child was not ideal. Some instructions were difficult for her to understand, such as looking in the camera, and the attention span is only limited. This child was not an actor, something that would be recommended for future recordings. It is probably easier to add more interaction and have the child act out certain types of behaviour if the child is a trained actor. This would also be a way to deal with privacy and ethical concerns, since this can be handled with a casting agency for example.

7 CONCLUSION

When answering the question ‘Can Virtual Reality technol-ogy be applicable for providing complementary training in the first phase of Parent-Child Interaction Therapy?’ the lit-erature review showed there is definite potential. First of all VR has shown to be effective in context of training and learning, which is the main goal of a complementary training application. The unique features of VR, mostly immersion, have a positive effect on learning and conceived realism.

Furthermore, research and real-world use of VR in behavior-and exposure therapy have proven its potential. Cost- effec-tiveness, high amount of control and limiting the role of external factors are the main reasons why VR based therapy is already being used in clinical practice. These reasons could be applied to the VR application for PCIT as well.

Lastly, the use of 360-degree footage is shown the be an effective way of conveying a virtual world. The degree of immersion is high when the footage is (converted to) a stereo-scopic image and combined with an HMD. Even using low-fidelity HMDs, such as Google Cardboard, have shown to offer a high degree of realism and immersion. This whilst still being relatively uncomplicated to use, and being less likely to generate simulator sickness, make it an attractive option for an application prototype.

In cooperation with therapists at De Bascule a final con-cept was drafted for an application that could help parents in the first phase of PCIT. This concept focuses on training three important Child-Directed Interaction Skills: labeled praises, reflections and behavior descriptions. Using A-Frame a pro-totype that runs on a wide variety of devices was successfully created. The prototype was evaluated by five field experts. These experts used the application, filled in a SUS question-naire and were questioned in a semi-structured interview.

In terms of usability the application scored very well. A SUS score of 88.5 is considered very high, with 68 being the average. All participants mentioned how simple the appli-cation was to use. Improvements can be made however by increasing volume and better text positioning. The appli-cation was deemed to be a good fit for the first phase of PCIT, and it addresses issues parents have during this phase. The videos represent special playtime in an immersive and realistic manner. The videos could be further improved by adding more interaction between viewer and child, and by adding some more content that also shows more challenging behaviour. However, after fixing the minor usability issues, most experts felt like the prototype was ready to be tested in clinical practice and showed great enthusiasm to do so.

The results from both the literature review and the eval-uation by experts have shown the potential of using VR in the first phase of PCIT. This provides convincing evidence to conclude that indeed virtual reality technology can be applied to provide complementary training in the first phase of PCIT. In order to confirm these results it is advisable to further fine tune the prototype and test the effectiveness in practice with a follow-up research.

ACKNOWLEDGMENTS

First of all I would like to thank Dr. Rob Belleman and Dr. Ramón Lindauer for initiating this project and providing me

with the opportunity to put such an innovative idea into practice. As my supervisor, Rob facilitated this project and gave me excellent feedback, for which I would like to thank him.

My thanks also go out to the staff and researchers at De Bascule who worked with me to evaluate the prototype. Spe-cial thanks go out to Dr. Mariëlle Abrahamse and Mirte Mos, who played a vital part in the ideation phase and forming the final concept. Mariëlle had an indispensable role throughout the project, helping me in many different aspects. For this I am very grateful.

Finally my thanks go out to my brother, sister-in-law and my cousin Lizzy. Without their patience, hospitality and great help I would not have been able to make the training videos.

REFERENCES

[1] A-Frame. [n. d.]. A-Frame Documentation. https://aframe.io/docs/0.8. 0/introduction/

[2] Mariëlle Abrahamse. 2015. Treating young children’s disruptive behav-ior problems: dissemination of an evidence-based parent management training program in the Netherlands. (2015).

[3] Ashfaq Amin, Diane Gromala, Xin Tong, and Chris Shaw. 2016. Immer-sion in cardboard VR compared to a traditional head-mounted display. In International Conference on Virtual, Augmented and Mixed Reality. Springer, 269–276.

[4] Page L Anderson, Elana Zimand, Larry F Hodges, and Barbara O Rothbaum. 2005. Cognitive behavioral therapy for public-speaking anxiety using virtual reality for exposure. Depression and anxiety 22, 3 (2005), 156–158.

[5] Jeremy N Bailenson, Nick Yee, Jim Blascovich, Andrew C Beall, Nicole Lundblad, and Michael Jin. 2008. The use of immersive virtual reality in the learning sciences: Digital transformations of teachers, students, and social context. The Journal of the Learning Sciences 17, 1 (2008), 102–141.

[6] Yanan Bao, Huasen Wu, Albara Ah Ramli, Bradley Wang, and Xin Liu. 2016. Viewing 360 degree videos: Motion prediction and bandwidth optimization. In Network Protocols (ICNP), 2016 IEEE 24th International Conference on. IEEE, 1–2.

[7] J Gayle Beck, Sarah A Palyo, Eliot H Winer, Brad E Schwagler, and Eu Jin Ang. 2007. Virtual reality exposure therapy for PTSD symptoms after a road accident: An uncontrolled case series. Behavior therapy 38, 1 (2007), 39–48.

[8] Benoit Bideau, Richard Kulpa, Nicolas Vignais, Sébastien Brault, Franck Multon, and Cathy Craig. 2010. Using virtual reality to analyze sports performance. IEEE Computer Graphics and Applications 30, 2 (2010), 14–21.

[9] Meredith Bricken. 1991. Virtual reality learning environments: poten-tials and challenges. ACM SIGGRAPH Computer Graphics 25, 3 (1991), 178–184.

[10] John Brooke. 2013. SUS: a retrospective. Journal of usability studies 8, 2 (2013), 29–40.

[11] John Brooke et al. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry189, 194 (1996), 4–7.

[12] Frederick P Brooks. 1999. What’s real about virtual reality? IEEE Computer graphics and applications19, 6 (1999), 16–27.

[13] Abbie Brown and Tim Green. 2016. Virtual reality: Low-cost tools and resources for the classroom. TechTrends 60, 5 (2016), 517–519. [14] Alan Bryman. 2016. Social research methods. Oxford university press.

[15] Timothy J Buker, Dennis A Vincenzi, and John E Deaton. 2012. The effect of apparent latency on simulator sickness while using a see-through helmet-mounted display: Reducing apparent latency with predictive compensation. Human factors 54, 2 (2012), 235–249. [16] Joshua Q Coburn, John L Salmon, and Ian Freeman. 2018. Effectiveness

of an Immersive Virtual Environment for Collaboration With Gesture Support Using Low-Cost Hardware. Journal of Mechanical Design 140, 4 (2018), 042001.

[17] Frederique Coelman, Willemine Heiner, Anke Breg, MariÃńlle Abra-hamse, Gonnie Albrecht, and Ramon Lindauer. 2014. Handleiding Parent-Child Interaction Therapy 2011. PI Research (Dec 2014). [18] Rae A Earnshaw. 2014. Virtual reality systems. Academic press. [19] Chris Fowler. 2015. Virtual reality and learning: Where is the

peda-gogy? British journal of educational technology 46, 2 (2015), 412–422. [20] Andreas Hebbel-Seeger. 2017. 360 Degrees Video and VR for Training

and Marketing within Sports. Athens Journal of Sports (2017). [21] Shih-Fong Huang, Po-Yi Tsai, Wen-Hsu Sung, Chih-Yung Lin, and

Tien-Yow Chuang. 2008. The comparisons of heart rate variability and perceived exertion during simulated cycling with various viewing devices. Presence: Teleoperators and Virtual Environments 17, 6 (2008), 575–583.

[22] Khadeeja Ibrahim-Didi. 2015. Immersion within 360 video settings: Capitalising on embodied perspectives to develop reflection-in-action within pre-service teacher education. (07 2015).

[23] Jason Jerald. 2015. The VR book: Human-centered design for virtual reality. Morgan & Claypool.

[24] Byeol Kim, Warren Schwartz, Danny Catacora, and Monifa Vaughn-Cooke. 2016. Virtual Reality Behavioral Therapy. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 60. SAGE Publications Sage CA: Los Angeles, CA, 356–360.

[25] Kwanguk Kim, M Zachary Rosenthal, David Zielinski, and Rachel Brady. 2012. Comparison of desktop, head mounted display, and six wall fully immersive systems using a stressful task. (2012).

[26] Eugenia M Kolasinski. 1995. Simulator Sickness in Virtual Environments. Technical Report. Army research inst for the behavioral and social sciences Alexandria VA.

[27] Merel Krijn, Paul MG Emmelkamp, Roeline Biemond, Claudius de Wilde de Ligny, Martijn J Schuemie, and Charles APG van der Mast. 2004. Treatment of acrophobia in virtual reality: The role of immersion and presence. Behaviour research and therapy 42, 2 (2004), 229–239.

[28] Merel Krijn, Paul MG Emmelkamp, Ragnar P Olafsson, and Roeline Biemond. 2004. Virtual reality exposure therapy of anxiety disorders: A review. Clinical psychology review 24, 3 (2004), 259–281.

[29] Dan MacIsaac. 2015. Google Cardboard: A virtual reality headset for $10? The Physics Teacher 53, 2 (2015), 125–125.

[30] Zahira Merchant, Ernest T Goetz, Lauren Cifuentes, Wendy Keeney-Kennicutt, and Trina J Davis. 2014. Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education 70 (2014), 29–40. [31] Jakob Nielsen. [n. d.]. Thinking Aloud: The #1 Usability Tool. https:

//www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/ [32] Jakob Nielsen. [n. d.]. Why You Only Need to Test

with 5 Users. https://www.nngroup.com/articles/ why-you-only-need-to-test-with-5-users/

[33] David Opriş, Sebastian Pintea, Azucena García-Palacios, Cristina Botella, Ştefan Szamosközi, and Daniel David. 2012. Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depression and anxiety29, 2 (2012), 85–93.

[34] Zhigeng Pan, Adrian David Cheok, Hongwei Yang, Jiejie Zhu, and Jiaoying Shi. 2006. Virtual reality and mixed reality for virtual learning environments. Computers & Graphics 30, 1 (2006), 20–28.

[35] Roy Pea, Michael Mills, Joseph Rosen, Kenneth Dauber, Wolfgang Effelsberg, and Eric Hoffert. 2004. The diver project: Interactive digital video repurposing. IEEE multimedia 11, 1 (2004), 54–61.

[36] Mark B Powers and Paul MG Emmelkamp. 2008. Virtual reality expo-sure therapy for anxiety disorders: A meta-analysis. Journal of anxiety disorders22, 3 (2008), 561–569.

[37] Wen Qi, Russell M Taylor II, Christopher G Healey, and Jean-Bernard Martens. 2006. A comparison of immersive HMD, fish tank VR and fish tank with haptics displays for volume visualization. In Proceedings of the 3rd Symposium on Applied Perception in Graphics and Visualization. ACM, 51–58.

[38] Albert Rizzo, Arno Hartholt, Mario Grimani, Andrew Leeds, and Matt Liewer. 2014. Virtual reality exposure therapy for combat-related posttraumatic stress disorder. Computer 47, 7 (2014), 31–37. [39] A Fleming Seay, David M Krum, Larry Hodges, and William Ribarsky.

2002. Simulator sickness and presence in a high field-of-view vir-tual environment. In CHI’02 Extended Abstracts on Human Factors in Computing Systems. ACM, 784–785.

[40] Sarah Sharples, Sue Cobb, Amanda Moody, and John R Wilson. 2008. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays29, 2 (2008), 58–69.

[41] Sherrill J Smith, Sharon Farra, Deborah L Ulrich, Eric Hodgson, Stephanie Nicely, and William Matcham. 2016. Learning and reten-tion using virtual reality in a decontaminareten-tion simulareten-tion. Nursing education perspectives37, 4 (2016), 210–214.

[42] Jaye Wald and Steven Taylor. 2000. Efficacy of virtual reality exposure therapy to treat driving phobia: a case report. Journal of behavior therapy and experimental psychiatry31, 3-4 (2000), 249–257. [43] Brooklyn Waters. 2015. Physics and Frame Rate: Beating

mo-tion sickness in VR. http://mtechgames.com/downloads/ PhysicsandFramerateBeatingmotionsicknessinVR.pdf

[44] Alan Wexelblat. 2014. Virtual reality: applications and explorations. Academic Press.

[45] Todd Williamson and Norihisa Suzuki. 2004. Virtual reality immersion system. US Patent App. 10/628,951.

[46] Jesse H Wright, Gregory K Brown, Michael E Thase, and Mon-ica Ramirez Basco. 2017. Learning cognitive-behavior therapy: An illustrated guide. American Psychiatric Pub.

[47] Yan Xu, Hyungsung Park, and Youngkyun Baek. 2011. A new approach toward digital storytelling: An activity focused on writing self-efficacy in a virtual learning environment. Journal of educational technology & society14, 4 (2011), 181.

[48] Yuichiro Yamashita and Nakajima Taira. 2016. Presentation Skills Training by Using a 360 Degree Camera. In EdMedia: World Conference on Educational Media and Technology. Association for the Advancement of Computing in Education (AACE), 1381–1384.

[49] S Yoganathan, DA Finch, E Parkin, and J Pollard. 2018. 360 virtual reality video for the acquisition of knot tying skills: A randomised controlled trial. International Journal of Surgery 54 (2018), 24–27. [50] Alison Zisser and Sheila M Eyberg. 2010. Parent-child interaction

PCIT VR prototype evaluation

Strongly Strongly disagree agree 1. I think that I would like to

use this system frequently 2. I found the system unnecessarily complex

3. I thought the system was easy to use

4. I think that I would need the support of a technical person to be able to use this system

5. I found the various functions in this system were well integrated

6. I thought there was too much inconsistency in this system

7. I would imagine that most people would learn to use this system very quickly

8. I found the system very cumbersome* to use

9. I felt very confident using the system

10. I needed to learn a lot of things before I could get going with this system

* Cumbersome: lastig, omslachtig, hinderlijk.

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

Questionnaire 1 Questionnaire 2 Questionnaire 3 Questionnaire 4 Questionnaire 5

Question Answer Score Answer Score Answer Score Answer Score Answer Score

1 4 3 4 3 4 3 5 4 5 4 2 1 4 2 3 1 4 1 4 1 4 3 3 2 5 4 5 4 4 3 5 4 4 3 2 2 3 2 3 2 3 1 4 5 4 3 4 3 4 3 4 3 4 3 6 2 3 1 4 1 4 1 4 2 3 7 4 3 5 4 5 4 5 4 5 4 8 2 3 1 4 1 4 1 4 1 4 9 4 3 5 4 4 3 5 4 5 4 10 1 4 1 4 1 4 1 4 1 4 Total 30 36 36 37 38 Score 75 90 90 92,5 95 Average score 88,5