A visual editor for creating non-linear timelines for PCIT-VR.

(1)

Bachelor Informatica

A visual editor for creating

non-linear timelines for PCIT-VR.

Emil van Veen

January 29, 2021

Supervisor(s): Robert Belleman

Inf

orma

tica

—

Universiteit

v

an

Ams

terd

am

(2)

(3)

Abstract

Parent-Child Interaction Therapy (PCIT) is an evidence-based form of therapy that involves working with parents and their young children that express disruptive behavioural disorders. In recent works, a virtual reality application was developed that allows parents to practice their newly acquired PCIT skills from their mobile phones, also known as PCIT-VR. Using simulations of real life therapy sessions, PCIT-VR shows 360 degree footage of real-life scenes that contain key events that parents are to respond to. A key aspect of these sessions are their use of non-linear timelines, meaning that each scene that is shown to the parent is determined by the way they respond during the key interaction moments.

In this thesis, I research and develop a web-based tool that aids therapists in creating non-linear timelines for PCIT-VR, while putting an emphasis on the usability of the user interfaces. A literature study is conducted in order to optimally design and visualize non-linear timelines, and usability principles are applied to ensure ease of use. The result is a well-functioning application that provides therapists with the all the required functionality, including a scenario and timeline editor, that is also compatible with the PCIT-VR Player. Maintainability metrics are gathered in order to analyze the state of the application, and show promising results. Going forward, I propose to research the possibilities of integrating the PCIT-VR Player into the editing application, and to look for ways to extend the project beyond PCIT-VR. Additionally, more usability testing should be applied to test and improve the overall user experience of the application.

(4)

(5)

4.3.2 Entity Identifiers . . . 35 4.4 Maintainability Evaluation . . . 35 4.4.1 API . . . 35 4.4.2 Front-End . . . 35 5 Results 37 5.1 Editing Application . . . 37 5.1.1 Functional Requirements . . . 37 5.1.2 Scenario Editor . . . 37 5.1.3 Timeline Editor . . . 38 5.2 Maintainability Measurements . . . 39 5.2.1 API . . . 39 5.2.2 Front-End . . . 40 6 Discussion 41 6.1 Application . . . 41

6.1.1 Design versus Realization . . . 41

6.1.2 Future Work . . . 41 6.2 Usability Testing . . . 42 6.3 Maintainability Measurements . . . 42 6.4 Application Testing . . . 43 6.5 Ethical Aspects . . . 43 7 Conclusion 45

A PCIT-VR Player JSON Specification 51

B Mockups 59

C Cyclomatic Complexity Front-End 61

(7)

CHAPTER 1

Introduction

Parent-Child Interaction Therapy (PCIT) is an evidence-based form of therapy that involves working with parents and their young children that express disruptive behavioural disorders. In this approach, a therapist coaches a parent interacting with their child from behind a one-way mirror, using a hearing aid to communicate. The goal of this form of therapy is to greatly im-prove the parent-child relationship, and to give parents the skills to develop better parent-child interaction patterns [25]. PCIT attempts to achieve this goal by allowing parents to master two sets of skills. Parents start with phase one: Child-Directed Interaction (CDI), which aims to improve the parent-child relationship by reinforcing appropriate behaviour and ignoring unde-sired behavior. This is done by teaching the parent how to praise appropriate behaviour, reflect on appropriate talk, and show enthusiasm. Inappropriate behavior is ignored completely during this phase. Once the parent shows to have mastered these skills, they continue to the Parent-Directed Interaction (PDI) phase. Here, the parent takes a more leading role, and learns how to set boundaries and manage the behavior of the child. This should ultimately improve child compliance and decrease disruptive behaviors.

PCIT therapists have observed that parents experience difficulty practicing their newly ac-quired skills outside of the therapy sessions, primarily due to the absence of a coach, and a change of environment [43]. This slows down the parents’ progress, as parents forget parts of what they have been taught during therapy sessions, ultimately decreasing the effectiveness of the therapy. To aid the parents’ learning process, a virtual reality (VR) application has been developed that can be used to reinforce the parents’ skills through their mobile phone [8, 38, 43]. Simulating a real life therapy session via VR, parents are shown 360 degree footage of real-life scenes, generally played out by actors, that contain key events that parents are to respond to. At key interaction moments, i.e. when the parent is supposed to react to a certain situation, the video freezes, and the parent is shown a question with a set of options on how to properly respond. The parent then chooses one of the options, which allows them to move on to the next scene. By repeating these sessions, the aim is to improve the learning experience of the parents, and to increase the effectiveness of the therapy over time.

A key aspect of the PCIT-VR application is non-linear timelines. In a non-linear timeline, each scene that is shown to the parent is determined by the option that they chose previously. As such, a practice session can take various paths based on how parents respond at the end of a given scene. This aspect is important, as it allows for replaying scenes where the parent picked an inappropriate option, and adds flexibility for the therapist to emphasise certain skills based on how the parent is performing. To illustrate, the therapist creates a sessions for practicing skills A and B. If the parent performs well in scenes for skill A, but less so in scenes for skill B, the therapist can design a path that allows the parent to practice more of skill B.

The use of non-linear timelines has been spearheaded by Vermeulen [43], and has been suc-cessfully implemented in the VR environment that the parents currently operate in. This has given therapists the possibility to create practice sessions themselves, including non-linear time-lines and key interaction moments. It does, however, leave much to be desired from a usability

(8)

standpoint. First, therapists have to create these timelines by manually editing JSON 1 files. This makes the creation of practice sessions labour-intensive and error-prone. Second, these JSON files have to be manually added to a remote server by the therapist to be accessible by the parent. This too is error-prone, and can have major consequences when done incorrectly, such as unintentional deletion of projects.

As part of a research collaboration between the University of Amsterdam and Levvel, I will research the PCIT-VR editor for its usability and effectiveness, focusing specifically on the creation of non-linear timelines. I will design and implement a web based Graphical User Interface (GUI) that allows therapists to create these non-linear timelines with ease, while ensuring the effectiveness and maintainability of the application. The work done in this thesis will therefore work towards making PCIT-VR more user friendly, and consequently more production-ready. The research question that follows is “What is the best possible way for designing and implementing a non-linear PCIT-VR timeline editor, while ensuring ease of use and maintainability?”.

In order to answer this question, I start by giving the theoretical context of the project (chapter 2). Here, I discuss the previous work done on the PCIT-VR project, followed by a deep dive into non-linearity, usability and maintainability. Next, I present the design considerations based on a set of functional requirements. These design considerations are then visualized via user interface mockups, which are used as the main reference during the implementation stage (chapter 3). The technical details of the implementation are covered in Chapter 4. The final chapters of the thesis (chapters 5, 6, and 7) present the results, followed by a discussion and conclusion.

(9)

CHAPTER 2

Theoretical Framework

Before proceeding to the design and implementation of the GUI for creating non-linear timelines for PCIT-VR, it is necessary to present some background information on the topic and clarify some of the terms. I start by presenting the history of the development of PCIT-VR (2.1). Here I talk about why PCIT-VR was created, what results it yielded, and what the current state of the application is. Next, I discuss the concept of non-linearity in the context of storytelling and training (2.2). I argue what benefits non-linearity can have and how they can improve the overall learning experience. In section 2.3, some background information will be given on the theory of usability. I discuss the principles that make up good usability design, and ways to go about incorporating these principles into the final design of the application. Finally, in section 2.4, I talk about ways for measuring the maintainability of a software project, and how they can be applied to this application.

2.1 Previous Work

2.1.1 PCIT-VR Prototype

PCIT-VR is a virtual reality application designed specifically for simulating interactive PCIT therapy sessions from a mobile device. It was first introduced by Schrijver [38] in a research collaboration with several PCIT therapists at Levvel, a center for child and youth psychiatry. The aim of this project was to allow parents that experience difficulties applying their newly acquired skills outside of therapy sessions to fully immerse in a virtual world that reenacts certain key PCIT handling techniques. It allows parents to do this from their own homes and in their own time, eliminating the need for a therapist to be present. The goal of their research was to measure whether reinforcing the learning experience via VR had a positive effect on the overall effectiveness of PCIT therapy.

An early prototype of PCIT-VR would allow parents to immerse in a virtual therapy session

by the means of 360° pre-recorded media. Each session contained an array of scenes of

real-life therapy sessions, simulating situations in which parents should apply skills they have been taught. To test these skills, each scene would be paused at certain key interaction moments, i.e. moments at which the acquired skills should be applied, and the parent would be presented with a question and a set of handling techniques, one of which is the most appropriate. Once the parent had played every scene, the session would reach its end.

The two main requirements for the prototype were cost effectiveness and accessibility. Any parent should be able to play therapy sessions without having to acquire any expensive VR gear. As such, a decision was made to target the development for Head Mounted Displays (HMDs) by the means of a smartphone, eliminating the requirement for said VR gear, and making it much more cost effective. Additionally, the application was designed to be web-based, making it possible for the parent to access the application straight from their browser. Being web-based also has benefits from a development perspective, as it makes the application largely operating system agnostic. Now, rather than having to develop the application for multiple platforms,

(10)

developers can target a variety of device types with a single application. Furthermore, with the

surge of web-based VR technologies, such as A-Frame1 and BabylonJS2, developers can create

such VR applications for the web with relative ease.

A downside of using smartphones as the main device is that it introduces a restriction on the degrees of freedom that the user has while using the application. Dedicated VR-gear usually allow for six degrees of freedom (6DoF), which includes rotational movement and translations along the x, y, and z axes [3]. Smartphones, on the contrary, only allow for rotational movement along all axes, limiting the user to three degrees of freedom (3DoF). This means that users are placed in a fixed position in the virtual environment, and can only explore the virtual environment via

head movements. However, it is shown that having three degrees of freedom using 360° media can

still be beneficial for educational purposes, as long as it allows for full immersion into a virtual world [45, 16, 2]. Ultimately, the PCIT-VR prototype was well received by therapists [38], and showed promising results in the effects on the overall experience of the therapy perceived by the parents. Further development was therefore made possible.

2.1.2 The Addition of Non-Linear Timelines

One of the main limitations of the prototype was its inability to impose reinforced learning on its users. Each session would consist of a predefined set of scenes, all in a fixed order. No playback of clips was possible, and the parents’ performance had no impact on the overall progression through the session. This limited the therapists’ flexibility to take into account the performance of the parent during a virtual therapy session. Rather, it should be possible to dynamically alter the subsequent sequence of scenes based on actions taken during key interaction moments. To accommodate for this, Vermeulen [43] introduced the concept of non-linear timelines to PCIT-VR. As they discovered, therapists prefer a tree-like structure (Figure 2.1) where scenes are connected via their interaction options. This allows for a branching functionality where each subsequent scene presented to the user is determined by the handling technique they deem most appropriate. Moreover, branching allows for replaying certain scenes that parents did not respond to correctly the first time, and gives therapists the opportunity to show a different set of scenes based on how the parent is progressing during the session.

Vermeulen [43] designed a new VR player specifically for playing therapy sessions based on non-linear timelines. It included branch-like behavior based on the parents’ interaction and allowed for tracking data for therapists to measure their overall performance. This branch-like behavior was implemented using a modular approach. Rather than being fixed within the application itself, therapists can design non-linear timelines themselves by creating and uploading configuration files containing all the necessary information to compose a session, such as clips, annotations and links. This configuration file will in turn be read by the VR player, which handles the logic and interaction of the session.

Due to lack of time, no graphical user interface was developed for the creation of non-linear timelines. As a result, therapists must create these definitions by hand and upload them to the server manually in order to make it accessible to the VR player, and ultimately their clients. This leaves much to be desired from a usability standpoint. First, editing these files without the correct tools and knowledge is a rather technical and error-prone process. The configuration files must adhere to a specific syntax and format, and little to no feedback is given when one of these requirements is not met. Additionally, since there is no visual representation of the configuration itself, users have little visibility on the status of the configuration [30].

Having to manually upload these files to a remote server also requires knowledge about the inner structure of the application’s folder hierarchy, and may have undesired side effects. Storing files in the wrong directory may lead to unexpected results where sessions become unreachable, and risks putting the folder structure in an undesired state. Allowing therapists to access the files may also lead to unintentional deletions of projects. Without any recovery mechanism in place, this could have disastrous consequences.

Overall, Vermeulen’s work shows a successful attempt to add support for non-linear timelines in PCIT-VR, with the resulting functionality of the VR player being very well-received by the

1_{https://aframe.io/}

(11)

Figure 2.1: Example of a non-linear timeline in PCIT-VR. Source: Vermeulen [43]

therapists [43]. Room for improvement can be found in the creation of non-linear timelines, which currently has an error-prone and confusing workflow.

2.2 Non-Linear Timelines

2.2.1 Non-Linear Storytelling

Non-linearity is a concept that is becoming more commonly used in storytelling, and breaks away from the sequential narrative stemming from classical storytelling [41]. Rather than prescribing a story-line that forces the consumer to follow along the story’s linear path, users are empowered to make their own choices along the way. As such, stories can have various endings based on the users’ interaction, with different paths taken during their journey. When combined with the rapid developments in digital storytelling, the personalized characteristics of non-linear sequences allow for a wide variety of applications [41]. In edutainment (educational entertainment), it allows for students to obtain a more personalized learning experience, rather than consuming a pre-defined and fixed stream of information [42]. As such, they can choose to opt in on learning more about particular subjects, while giving them the freedom to move on to something else if they find that subject more interesting. This has been shown to give a more positive effect on the overall mastery of the subject at hand [36].

Non-linearity is also a concept that can be beneficial in simulation-based training applications

[11, 18, 22]. In simulated-based training, the trainer aims to direct the training towards a

particular training objective, and wants to have the trainee exercise a particular set of skills depending on the trainee’s existing capabilities. It is therefore necessary to allow the trainer some form of control over how a scenario is generated, such that it can reflect the trainer’s preferences for various training objectives [22]. In order to do so, a mechanism must be put in place that allows for branching based on the trainees decision making, thereby giving the trainer the control over the direction of the trainees’ training.

(12)

2.2.2 Visualizing & Authoring Interactive Non-Linear Media

Non-linear timelines have certain characteristics in the digital media domain, such as non-linear structure, alternative playback paths, choice elements, and the influence on the order of scenes [27]. These characteristics often make it more difficult to produce interactive non-linear media than the traditional, linear media. This stems from the fact that more material is required in order to compose a story, and a consistent representation of the project must be present in order to efficiently create such media. Additionally, workflows to create such media are often iterative, meaning that there is a strong emphasise on previewing and testing a produced story while under construction [14].

Various tools have been created for authoring and managing interactive non-linear media, specifically in the context of video [26, 27, 10]. Most of these tools share common characteristics and functionalities, and aim to allow for combining elements that make up an interactive non-linear story. As such, there has to be a mechanism in place for managing and keeping the story structure [10, 27], including its scenes, assets, story logic and timelines. More importantly, the application should allow for editing of the non-linear story. An important requirement for editing a non-linear story is that it should give the user a representative, logical, and visual representation of the flow of the story. A common consensus on how present this, is by the means of a directed graph [27, 4]. A directed graph gives a visual representation of the components in a non-linear story and the sequence of ordering amount those components via their actions. Furthermore, this level of abstraction allows for eliminating the need for programming skills on the users’ end. Users can simply create non-linear timelines by adding nodes to the graph, i.e. adding scenes to the story, and connect these to form a non-linear story [10].

2.3 Usability

Usability is a concept that applies to most, if not all, software production that involves user interfaces (UIs). The constant growth of computer users has made it essential to put an em-phasis on usability, especially since a large group of users are users who do not have computer backgrounds or extensive working experience with computers [7]. This lack of experience can hinder the users’ ability to learn the software, and make it difficult for them to create the build conceptual model with respect to how the software is intended to be used [31, 7]. It is therefore crucial to design an application and its intended use with the user in mind [32, 12].

2.3.1 Putting the User First

Gould and Lewis [12] propose three key principles for achieving optimal usability: 1) early focus on users and tasks, 2) empirical measurement, and 3) iterative design. The first principle, early focus on users and tasks, states that designers must have a good understanding of what the future users of their software will be, and what tasks must be performed. This understanding can be achieved by studying the users, and figuring out their characteristics and goals. The goal of the second principle, empirical measurements, is to use simulations of the future software to find any potential issues early in the development process. This principle entails testing the simulations with real users and studying their reactions and performance. The final principle, iterative design, states that when issues are found in the testing process, they must be fixed. This has the effect of developing a development cycle, where every added feature is tested, measured and redesigned appropriately.

Nielsen [31] presents the usability engineering life cycle, which is an extension to the key principles proposed by Gould and Lewis [12]. In the usability engineering life cycle, there are usability activities for three of the main phases of the software development process: before, during and after. In the before phase, it is important to understanding the targeted users and their tasks. The goal is to not rush into the design, but to first figure out what the capabilities and needs are of the users. This way a pre-defined set of limits can be defined for the complexity of the future UI. An addition to this phase compared to Gould and Lewis’s work, is that the context in which the user operate is considered to be highly important, and should be taken

(13)

Goal Description

Learnability A user interface should be easy to learn.

Efficiency The user must be able to efficiently use the system once learned.

Relearnability Users should be able to return to the system without having to relearn it.

Frequency of Errors (Serious) Errors made by users should be prevented as much as possible.

Satisfaction Users should be satisfied using the system

Table 2.1: The five usability goals proposed by Nielsen [31]

into consideration when designing the UI. Additionally, Nielsen proposes the five usability goals (Table 2.1), which define the characteristics and goals that a design should aim to achieve.

The next phase of the life cycle is the design phase, during which the designer should aim to come to a releasable product. Prototypes are essential during this phase, as it allows the designer to test the future UI quick and often. Once a satisfactory design has been prototyped and implemented, the designer can continue to the post-design phase. Here, the designer should continue gathering data for future versions of the application.

Norman [33] recommends the human-centered design philosophy (HCD). In this approach, human needs, capabilities and behavior form the central elements that make up a design. This means starting with a good understanding of the future end-user, and the needs for the design in order to solve their problems. This can be achieved by observation. The basic idea behind this philosophy is that the users themselves often are unaware of their needs and difficulties that they are encountering, and that designers should unveil these difficulties and needs by observing and testing the users.

2.3.2 User Interface Principles

Designers should design and evaluate user interfaces based on a certain set of pre-defined rules and principles that give the designer a direction for creating usable systems [23]. Over the years, various frameworks have been developed that provide these rules.

Nielsen [30] proposes ten general principles for user interface design: 1) Visibility of system status; 2) Match between system and the real world; 3) User control and freedom; 4) Consistency and standards; 5) Error prevention; 6) Recognition rather than recall; 7) Flexibility and efficiency of user; 8) Aesthetic and minimalist design; 9) Help users recognize, diagnose, and recover from errors; 10) Help and documentation. These principles put an emphasis on simplicity and error prevention, and should aid the designer in achieving the goals presented in Table 2.1.

Shneiderman [39] proposes similar principles in their eight golden rules for interface design: 1) Strive for consistency; 2) Seek universal usability; 3) Offer informative feedback; 4) Design dialogs to yield closure; 5) Prevent errors; 6) Permit easy reversal of actions; 7) Keep users in control; 8) Reduce short-term memory load. Though principles 1, 3, 5, 6, 7, 8 overlap with the principles proposed by Nielsen, Shneiderman also addresses of how having concrete points of accomplishment and completion in principle 4, allowing for a sense of relief and preparation for the next next sequence of actions. Conversely, Nielsen stresses the importance of well-structured documentation, as well as using words and concepts familiar to the user.

For the task at hand I have selected the most relevant user interface principles and elaborate on these below.

Ease of Use

It is important to avoid user doubt by using proper standards and consistency. Words and actions should be context-agnostic; identical terminology should be used in prompts, menus and action buttons. Consistent font, layout and color should also be employed throughout to create consistent views. Moreover, language should not introduce any ambiguity, but rather speak the users’ language. Information should appear in natural and logical order, creating a flow that is intuitive to the user.

Both Nielsen [30] and Shneiderman [39] emphasise reducing the short-term memory load of the users. Users should not have to carry information with them from one view to another, but

(14)

the view should rather provide users with all the information they require. As such, every view should provide the user with a proper indication of the current state of the system, and make all functionality required to cater the users’ needs available.

Error Prevention

Errors should be eliminated as much as possible, and no action that the user is allowed to perform should put the application in an invalid state. When a user does make a mistake, the system should provide the user with clear instructions on how to recover from it [39].

According to Nielsen [30], error prevention can be implemented in two ways. After identi-fying error-prone conditions in the application, the designer can opt to either eliminate these conditions, or present the user with a confirmation screen before performing the action. This confirmation screen should make the user aware of the dangers of performing an action.

Efficiency

Nielsen [30] stresses limiting the information presented to the user to only that is required at a certain point of time. This not only minimizes the user interface, but also makes the important information more visible. Users are therefore less likely to miss important information or actions, ultimately improving their efficiency. Furthermore, a difference should be made between novice and advanced users. Customization should be made available to the user, allowing them to tailor frequent actions by adding shortcuts or custom actions.

2.4 Measuring Software Maintainability

Generally, there are two major phases in the software life cycle: software development and software maintenance. Chen and Huang [6] state that about 66% of the total software life cycle is dedicated to the latter, while only 2.7% of IT specialists considered software maintenance to be effective and efficient. This goes to show that software maintenance is a difficult process.

Various models for measuring the maintainability of software have been proposed over the years [29, 15, 1, 34]. These models generally combine various characteristics of software in order to determine its maintainability, including the program size, the amount of commented lines, and the percentage of unit test coverage, but may vary in their extensiveness. Some models allow for higher analyzability by scoring each characteristic individually [15], whereas others combine each individual characteristic into a single score [34]. For the task at hand, I have selected two metrics that are commonly used. I elaborate on these metrics below.

2.4.1 Cyclomatic Complexity

Cyclomatic Complexity (CC) is a software metric used to measure the complexity of a program, and is determined by the number of linearly independent graphs through the program’s source code. Introduced by McCabe [24], the measurement is derived from the program’s control-flow graph, a directed graph comprised of all the program’s code blocks, and edges between the blocks if control may pass from the first to the second block. An example of such a control-flow graph is given in Figure 2.2. More formally, the McCabe CC is defined as

M = E − N + 2P where

• E = the number of edges in the graph. • N = the number of nodes in the graph. • P = the number of connected components.

The ranking scheme for CC is defined in 2.2. Watson, Wallace, and McCabe [44] state that the score should not be any higher than 10 for any block of code in order to keep the complexity of the program sufficiently low.

(15)

1 2 3 4 6 7 8

Figure 2.2: Example of a control-flow graph (right) for piece of python code (left). The nodes are numbered according to corresponding line number in the code.

CC score Rank Risk

1 - 5 A low - simple block

6 - 10 B low - well structured and stable block

11 - 20 C moderate - slightly complex block

21 - 30 D more than moderate - more complex block

31 - 40 E high - complex block, alarming

41+ F very high - error-prone, unstable block

Table 2.2: Cyclomatic Complixity rankings.

2.4.2 Maintability Index

The maintainability index is a software metric introduced by Oman and Hagemeister [34] in 1992, that measures the maintainability of the program’s source code. The index is calculated based on the following factors: Source Lines of Code (SLOC), the Cyclomatic Complexity, and Halstead Volume. The formula for calculating the Maintainability Index (MI) is defined as

MI = max[0, 100 ·171 − 5.2 ln V − 0.23G − 16.2 ln L + 50 sin(

√ 2.4C)

171 ]

where

1. V = the Halstead Volume 2. G = the Cyclomatic Complexity 3. L = the Source Lines of Code 4. C = the percent of comment lines

Each component is calculated on the function level [35], and then averaged per module. A more detailed description of the Halstead Volume and Source Lines of Code are given below. The rankings for the Maintainability Index can be found in Table 2.3.

Halstead Volume

The Halstead Volume is one of the collection of metrics introduced by Halstead et al. [13]. It is a metric that measures the size of the program based on the four scalars listed below

(16)

MI score Rank Maintainability

100 - 20 A High

19 - 10 B Medium

9 - 0 C Low

Table 2.3: Maintainability index rankings. Source: https://docs.microsoft.com/en-us/

archive/blogs/codeanalysis/maintainability-index-range-and-meaning

• n1 = the number of distinct operators

• n2 = the number of distinct operands

• N1 = the total number of operators

• N2 = the total number of operands

Based on these scalars, the vocabulary (n) and length (N ) of the program are calculated as follows:

n = n1+ n2

N = N1+ N2

Finally, the vocabulary and length are combined to calculate the volume

N · log₂(n)

:

Source Lines of Code

SLOC is a popular metric to define the program or module size [35], but its definition may differ. Some definitions include commented lines, whereas others don’t. A distinction is also made between physical lines of code, and logical lines of code. Logical lines of code count only the lines that have any other content than just a whitespace or comment. Physical lines of code count all the lines, regardless of their contents. In the MI, the physical lines of code are used.

(17)

CHAPTER 3

Design

This chapter lays out the design considerations regarding the functional and usability require-ments for the development of a visual non-linear timeline editor for PCIT-VR. I start by dis-cussing the functional requirements (3.1), followed by a deep dive into the usability of the appli-cation (3.2). These considerations I then show through a number of mockups (3.3), that will be used as the main reference during the development stage. Lastly, I address the security features that must exist in order to prevent unauthorized access (3.4).

3.1 Functional Requirements

The main requirement of the application is that it allows therapists to easily create, manage, and combine all individual entities that make up a non-linear timeline. The application should be considered a single source of truth for the therapists, and contain all functionality required to avoid the current multi-tool setup discussed in section 2.1.2. The application should not make any assumptions regarding the technical capabilities of therapists, but rather provide intuitive interfaces that allow them to perform their required actions. Therefore, the application should abstract away from any technical specifications of non-linear timelines.

Before we design the interfaces for the application, it is important that we first identify all the separate entities that form these non-linear timelines. By breaking up these entities and allowing the end user to focus on each of them separately, we can make editing these timelines simpler and more user friendly. Additionally, this makes the application modular by allowing for reusability of each individual component.

In collaboration with therapists at Levvel, I have identified the following four main entities that form non-linear timelines:

1. Assets: 360° media of therapy sessions

2. Scenes: A combination of 360° media and annotations

3. Scenarios: A collection of scenes and links 4. Timelines: A collection of scenarios

These entities are not unrelated, but rather have a hierarchical structure; each higher level entity is made up from a combination of lower level entities. I discuss each individual entity more in depth in the upcoming sections.

3.1.1 Assets

Assets are a set of short, pre-recorded 360° media that form the cornerstone of PCIT-VR.

Sim-ulating real therapy sessions, these media are what make up the environment that the parents operate in, and allow them to fully immerse into a virtual world using head mounted displays.

(18)

For therapists, it is important that they can upload and manage these media. The application should enable the therapist to upload videos directly from their local machine, without requiring any intermediate steps or third party tools. Once the media is no longer required, the application should allow for safe and easy deletion of media. Safety is important here, because we want to prevent any errors resulting from actions performed by therapists. Furthermore, once uploaded, the media should be made available for the therapist to be used throughout the entire process of creating non-linear timelines.

Concretely, the therapists should be able to perform the following actions: 1. Upload: Therapists should be able to create assets

2. Delete: Therapists should be able to delete assets when they are no longer being used.

3.1.2 Scenes

Scenes are a combination of 360° media and annotations, and form the interactive content that is

provided to a parent during therapy sessions. As discussed previously, a scene will freeze at a key interaction moment, and a question will be presented along with a set of options. To illustrate,

parents are shown 360° video of a girl playing with tower made up from blocks. Eventually, the

tower falls over and the girl becomes angry. At this point it is time for the parent to respond, and a question is shown asking the parent how to appropriately handle the situation.

It is important that the application allows therapists to easily create these scenes. This

in-cludes selecting the 360° media to be used, and defining when and what annotations are presented.

Concretely, therapists should be able to perform the following actions: 1. Create: Therapists should be able to create scenes

2. Delete: Therapists should be able to delete scenes when they are no longer being used. 3. Annotations:

(a) Create: Therapists should be able to create annotations, defining the key interaction moments. This includes specifying the question, and an arbitrary number of options that go along with it. When creating an annotation, this should automatically be linked to a timestamp.

(b) Delete: Therapists should be able to delete annotations from a scene.

3.1.3 Scenarios

Scene A 1 2 Scene B 1 2 Scene C 1 2 Scene D 1 2 3 Scene E 1 2

Figure 3.1: Example of a scenario as a collection of scenes.

Scenarios are a collection of storylines formed by a set of scenes that are linked via their annotation options. A visual example of this is given in Figure 3.1. In this figure we see a collection of scenes, labeled from A to E, and a set of options for each of the scenes, represented

(19)

by circles at the edges. Links between are represented by arrows with options as their source and scenes as target. These links are what gives PCIT-VR its non-linear characteristics, and scenarios are therefore to be considered an essential part of the PCIT-VR project. As such, it is important that the application provides therapist with clear and intuitive interfaces for the creation of scenarios that account for difficulties discussed in 2.2.2. An emphasis will be put on providing the end user with interfaces that show clear representations of scenario and their flows. Additionally, all the scenes that are included in the scenario should be displayed in a way that makes them easily identifiable.

The following actions must be present for the creation of scenarios: 1. Create: Therapists should be able to create new scenarios

2. Delete: Therapists should be able to delete scenarios when they are no longer required. 3. Validate: Therapists should be able to validate scenarios for correctness.

4. Scenes:

(a) Add Scene: Therapists should be able to add new scenes to a scenario (b) Link Scenes: Therapists should be able to link scenes via their annotations.

(c) Unlink Scenes: Therapists should be able to unlink scenes.

(d) Select Start Scene: Therapists should be able to select what scene is the starting point of the scenario.

3.1.4 Timelines

The highest level entity; timelines are a collection of scenarios. Based on the therapists’ desires, a decision has been made to make timelines a separate entity, as it allows for ordering of scenarios. The order of scenarios can be sequential, i.e. scenarios are played in a fixed, pre-defined order, or randomized, meaning that the order of scenarios is different each time a therapy session is initiated. This is important for the therapists, as it avoids the possibility for end users to pattern match the answers to key interaction moment based on their previous sessions.

Clear interfaces must be provided to the therapists to allow them to create and manage timelines, including adding and ordering scenarios. In terms of actions, the following must be made available to therapists:

1. Create: Therapists should be able to create new timelines 2. Delete: Therapists should be able to delete timelines

3. Define Order Type: Therapists should be able to define whether the scenarios should be shown in random or sequential order.

4. Scenarios:

(a) Add: Therapists should be able to add scenarios to a timeline. (b) Delete: Therapists should be able to delete scenarios to a timeline.

(c) Order: Therapists should be able to order scenarios as they please. 5. Parents:

(a) Link: Therapists should be able to link parents to a timeline. (b) Unlink: Therapists should be able to unlink parents to a timeline.

6. Export: Therapists should be able to export a timeline in a format that can be read by the PCIT-VR player

(20)

3.2 Usability

Usability principles should be applied to allow for efficient and accessible usage of the application. In this section, I present usability considerations made based on the user interface principles discussed in section 2.3.2 and show these considerations through a number of mockups.

3.2.1 Creating Non-Linear Timelines

The application allow the end-user to focus on the contents of creating non-linear timelines, rather than on its technical details. The user interface should therefore provide therapists with a clear and appropriate representation of non-linear timelines and their flows, and allow for intuitive editing. Based on the literature discussed in section 2.2, I decided to visualize and edit non-linear timelines as interactive graph-like structures.

There are two main ways to present the graph-like structures: horizontally or vertically. Though we are more often presented with vertically oriented graphs, horizontal orientation is preferred in this situation [27]. The aspect ratio of computer screens are generally greater in width than height, allowing for more optimal use of the screen surface. Additionally, horizontal orientation allows for reading from left to right. This has its benefits. First, people are used to read from left to right. Second, since essentially the graph is representing what scenes are shown at certain points in time, a horizontal orientation creates a more natural flow.

A great example of interactive non-linear timeline creation and editing in the commercial

do-main is WarpVR1. Their editor provides the clear view and flow that PCIT-VR requires (Figure

3.2), and uses the horizontal orientation for the graph-like structure that is preferred. Addi-tionally, their drag-and-drop functionality makes for intuitive editing. As such, I use WarpVR’s editor as the main inspiration for the non-linear timeline editor for PCIT-VR.

Figure 3.2: Horizontal representation of a non-linear timeline. Source: WarpVR https://www. youtube.com/watch?v=2CnKKbeStZA&ab_channel=WarpVR

3.2.2 Feedback & Error Prevention

To prevent user errors and confusion, it is important that appropriate feedback is given to them after performing an action. Based on the action performed, different types of feedback can be

(21)

given. For example, whilst the user is uploading a video the application can present a progress bar (Figure 3.3a), reassuring the user that the server is working on their task. Optionally, this progress bar could indicate the amount of work that already has been done, and how much work is left still. This would give the user an additional time indication.

Another example is to present the user with an alert banner (Figure 3.3b) after they have done some management action, such as deleting or adding a scene. Once an action has been carried out, a banner will pop up to notify the user about the outcome of their action. This banner will have a different color and icon combination based on the status of this outcome, e.g. red with an error icon when the action failed or green with a check mark icon when the action succeeded. Besides indicating the status, the banner should also provide a short and concise description of the status.

Error prevention can be implemented by asking for the user’s confirmation before carrying out a particular action, for example before the deletion of some scene. By using a dialog (Figure 3.3c), we can notify the user about any possible consequences of the action, and make sure that the user is willing to go through with the action.

(a) Example of a progress bar. Source: Material-UI https://material-ui.com/components/progress/

(b) Example of an alert banner. Source: Material-UI https://material-ui.com/components/alert/

(c) Example of a dialog. Source: Material-UI https: //material-ui.com/components/dialog/

Figure 3.3: Three ways to provide visual feedback to the user.

3.3 Mockups

I have created a number of mockups that are based on the usability considerations discussed in section 3.2, and will act as the main inspiration for the development of the application. In this section I discuss a subset of these mockups in more detail. Additional mockups can be found in Appendix B.

3.3.1 Editing Scenarios

The interface for editing scenarios, figure 3.4, is similar to that of WarpVR (figure 3.2). Users are presented with an overview of all the created scenes, and can add a scene to a scenario by a single click on an action button. Once added, a scene is visualized as a block consisting of the title and description of the scene. Circles are placed on the edges of the block to represent the options that can be linked from. Moreover, each block contains actions that can be performed on the scene in the context of scenario-editing, such as deleting, copying and editing. Linking scenes can be done by dragging and dropping an arrow from one of the options from the source scene to the block of the target scene. Once linked, the arrow is persistent and will continue to be displayed. Links can be removed by simply clicking on the arrow followed by pressing the escape key.

(22)

A dedicated view is provided for editing a scene. Using this view, users can change any meta information of the scene, e.g. the title and description, and add and manage annotations. To allow for iterative development of scenes, users are provided a preview. This preview is an

interactive 360° media video, and gives an indication of how the scene will be shown to the

end-user during a session.

As with scenes, an iterative development has also been enabled for scenarios. A preview of

a scenario can be started with a single click on a button. Again, an interactive 360° video is

presented to the therapist. Annotations can be used to navigate through the scenario timeline, making sure that all paths are set up correctly.

PCIT-VR A B Scenario A Welkom, Gebruiker Opslaan Tijdlijn Afspelen PCIT-VR Editor   Geuploade Video's Menu Sluiten  Naam: clip1.mov Tijdsduur: 38 sec

Toevoegen aan tijdlijn

Naam: clip2.mov Tijdsduur: 1 min 02 sec

Video Uploaden Naam: clip1.mov Tijdsduur: 38 sec Tijdsduur: 32 sec Video: clip1.mov bewerken   

users can upload videos by clicking on this button

clicking on this button will add a scenario to the timeline. A modal will pup up that allows users to configure scenario

PCIT-VR A B Scene A Welkom, Gebruiker Opslaan Tijdlijn Afspelen PCIT-VR Editor   Geuploade Video's Menu Sluiten  Naam: clip1.mov Tijdsduur: 38 sec

Video Uploaden Naam: clip1.mov Tijdsduur: 38 sec Tijdsduur: 32 sec Video: clip1.mov bewerken    A B Scene B Tijdsduur: 38 sec Video: clip2.mov bewerken   

1. Users can link two scenario's by clicking on one of the options 2. User can click on the scenario that it wants to link it to 3. An arrow will indicate that the two scenario's are now linked

PCIT-VR A B Scene A Welkom, Gebruiker Opslaan Tijdlijn Afspelen PCIT-VR Editor   Geuploade Video's Menu Sluiten  Naam: clip1.mov Tijdsduur: 38 sec

Video Uploaden Naam: clip1.mov Tijdsduur: 38 sec Tijdsduur: 32 sec Video: clip1.mov bewerken A C Scene B Tijdsduur: 20 sec Video: clip3.mov dupliceer B A C Scene C   Tijdsduur: 20 sec Video: clip3.mov B Scene D Tijdsduur: 20 sec Video: clip3.mov           click on edit to edit the scenario

clicking on the duplicate video lets the user duplicate a scenario click on close menu button

PCIT-VR A B Scenario A Welkom, Gebruiker Opslaan Tijdlijn Afspelen PCIT-VR Editor   Geuploade Video's Menu Sluiten  Naam: clip1.mov Tijdsduur: 38 sec

Video Uploaden Naam: clip1.mov Tijdsduur: 38 sec Tijdsduur: 32 sec Video: clip1.mov bewerken A C

Scenario BTijdsduur: 20 sec Video: clip3.mov dupliceer B A C Scenario C   Tijdsduur: 20 sec Video: clip3.mov B Scene B Tijdsduur: 20 sec Video: clip3.mov           Scene Toevoegen 0:00 0:38 Opslaan Annuleren Titel:Scene A Clip   start  einde Annotaties

Er zijn nog geen annotaties....

a modal will pop up that lets the user edit the scenario

Klik op de tijdlijn van de video om een annotatie toe te voegen users can use slider to trim the clip

Selecteer Media

Figure 3.4: Mockup of the scenario editor

3.3.2 Feedback

If a message is related to a scene, as seen in Figure 3.5 in the scenario editor, the border of the scene in the scenario graph will change color, allowing for visual recognition. A tooltip will become present above the scene, indicating the problem that has occured. To allow for easier development, scenes will stay highlighted until the problem has been resolved.

(23)

PCIT-VR A B Scene A Welkom, Gebruiker Opslaan Tijdlijn Afspelen PCIT-VR Editor   Geuploade Video's Menu Sluiten  Naam: clip1.mov Tijdsduur: 38 sec Toevoegen aan tijdlijn

Naam: clip2.mov Tijdsduur: 1 min 02 sec Toevoegen aan tijdlijn 

Video Uploaden Naam: clip1.mov Tijdsduur: 38 sec Tijdsduur: 32 sec Video: clip1.mov bewerken A C Scene B Tijdsduur: 20 sec Video: clip3.mov dupliceer B A C Scene C   Tijdsduur: 20 sec Video: clip3.mov B Scene D Tijdsduur: 20 sec Video: clip3.mov          

1. User removes link

PCIT-VR A B Scene A Welkom, Gebruiker Opslaan Tijdlijn Afspelen PCIT-VR Editor   Geuploade Video's Menu Sluiten  Naam: clip1.mov Tijdsduur: 38 sec Toevoegen aan tijdlijn

Naam: clip2.mov Tijdsduur: 1 min 02 sec Toevoegen aan tijdlijn 

Video Uploaden Naam: clip1.mov Tijdsduur: 38 sec Tijdsduur: 32 sec Video: clip1.mov bewerken A C Scene B Tijdsduur: 20 sec Video: clip3.mov dupliceer B A C Scene C   Tijdsduur: 20 sec Video: clip3.mov B Scene D Tijdsduur: 20 sec Video: clip3.mov           Onbereikbaar  

Een of meerdere scenes in het scenario zijn onbereikbaar



Startpunt Startpunt

2. Banner is shown indicating a warning

2. The affected scenes become highlighted

Figure 3.5: Mockup that showcases the visual validation of scenarios.

3.4 Security

Requirements

Due to the application containing sensitive personal information, European legislation states that it is required to prevent unauthorized access to the PCIT-VR application [9]. No entity should be allowed access to the application other than the therapists and parents associated to the PCIT-VR project. An authentication and authorization mechanism must therefore be put in place to restrict access to third party entities.

Restricting access entails that no third party entity should be able to retrieve any information stored in the application, such as user information, scenario’s and analytical data. As such, valid credentials much be provided when attempting to retrieve any data, and functionality must be developed to validate these credentials. Moreover, different types of users should have different types of permissions. Parents should be allowed to access timelines that they are assigned to, but not be allowed to retrieve any data from other parents. Therapists should have the permissions to retrieve any information regarding therapy sessions and parents that are specifically under their care.

The therapists at Levvel have expressed a desire to find an authentication solution for which parents do not require to log in before starting a session. This way, parents do not have to maintain their credentials, allowing them to focus solely on the session. Vermeulen [43] researched various technologies that allow us to implement this type of authentication and authorization in their thesis. These technologies include session cookies, OAuth 2.0 and JSON Web Tokens (JWTs). Ultimately, Vermeulen recommended using JWTs, due to their flexibility and stateless nature. These characteristics make it possible to provide parents with a token without them having to sign in, thus fulfilling the therapists’ desires. In the next section we will dive deeper into the workings of JWTs, and what allows them to be flexible and stateless.

JSON Web Tokens

JWT2is an open standard that defines a compact and self-contained way for securely transmitting

information, usually called claims, between parties using JSON objects [20]. All objects are digitally signed using a symmetric or asymmetric key, making them trustworthy and verifiable. Due to its relatively small size, JWTs can be sent over HTTP as a header or cookie. This, in combination with the digital signing, yield the possibility for JWTs to be stateless. Being a stateless mechanism has its benefits. First, no session management is required, as the JWT can contain all the information about the entity required to do both authentication and authorization. On an incoming request, the server can simply verify that the token contains all the necessary information and that it has been signed correctly. Second, JWTs are portable. Since JWTs can be passed as request headers, they are not limited to a single backend as long as the backend

2

(24)

knows how to verify the token.

A well-formed JWT consists of three concatenated Base64url-encoded strings [21], separated by dots:

1. Header. Contains metadata about the token itself, and usually consists of two parts: the type of token and the signing algorithm used.

2. Payload. Contains the claims. Claims can be statements about the entity, e.g. the user, and any additional data that the parties want to share.

3. Signature. The signature of the combination of the header and the payload separated by dots. Can be used to verify that the token has not been altered while being sent.

An example of this can be found in Figure 3.6.

(a) Decoded JWT structure (b) Encoded JWT structure

Figure 3.6: The structure of a JWT using the symmetric key my-secret-key. JWTs are com-posed of a header (red), payload (purple) and signature (blue), separated by dots.

(25)

CHAPTER 4

Methodology

This chapter presents the implementation details for the new editing application for the PCIT-VR project. I start by presenting the architecture of the application (4.1), where I briefly discuss the motivation for the technologies used, followed by a high-level overview of the application’s general architecture. Next, I dive deeper into technical details of certain parts of the application (4.1.4). Afterward, I examine the changes I make to the existing PCIT-VR player application in order to make it compatible with the new editing application (4.2), resulting in a flow as seen in Figure 4.1. Finally, I take a look at the system’s security mechanisms (4.3), followed by the evaluation techniques used to measure the maintainability of the new editing application (4.4).

2. Upload Assets 3. Create Scenes 4. Create Scenario

Generated viewing link

5. Create Timeline Editing Application

Viewing Application

2. Retrieve Export 3. Play Timeline

Therapist 1. Login

Parent 1. Authenticate 4. Push Analytics

Figure 4.1: The flow of the PCIT-VR.

4.1 Editing Application

4.1.1 Technologies Motivation

The PCIT-VR editing application is a web-based single page application (SPA) developed with

TypeScript1, React.js2 and Material-UI3. An SPA is a web application that works inside the

browser and does not require page reloads during use. This has its benefits. First, due to the elimination of reloads, the page will feel fast and responsive. All the resources are only loaded once throughout the lifespan of the application, and thus uses less bandwith. Second, it allows for easy debugging, due to the wide range of available developer tools and easy network monitoring. TypeScript is a language which builds on JavaScript by adding support for optional static type definitions. Due to it being a superset of the JavaScript language, native JavaScript modules

1

https://www.typescriptlang.org/

2_{https://reactjs.org/}

(26)

is seen as valid TypeScript code, making it highly flexible. Furthermore, due to the fact that TypeScript transcompiles to JavaScript, it is compatible with any JavaScript application.

React.js is an open-source, declarative, and flexible JavaScript library dedicated to building user interfaces. User interfaces are composed of smaller, isolated entities, called components, which allow for modularity and separation of concerns. Moreover, React.js has its own language labeled JSX, which is an extension to the JavaScript language that allows programmers to write code using an XML-type syntax. React.js was chosen over other frameworks, such as VueJS and Svelte, because of its flexibility, ease of use, and maintanability.

Material-UI is a JavaScript framework designed specifically for React.js, and provides an array of pre-built components. The visuals and interaction of these components are based on Material Design, a design system created by Google to aid designers in creating high-performance visuals [17]. Perhaps more importantly, Material-UI comes with various built-in usability features that can be used throughout the application, such as visual feedback, intuitive interaction, and intuitive navigation components.

The application programming interface (API) is developed using a combination of Flask

and PostgreSQL. Flask4 is a python-based micro web application framework, allowing for fast

development, modular design and easy debugging. With HTTP request handling functionality built in and easy connectability with various database systems, Flask is considered the most optimal framework for this application.

PostgreSQL5_{is an open source relational database system with tens of years of active}

devel-opment. PostgreSQL has its benefits compared to its competitors, including MySQL, Oracle and SQL Server. PostgreSQL is open-source and free to use, and it is fully ACID compliant, meaning that database transactions are guaranteed to be fully completed, and in a timely manner.

4.1.2 Architecture

A high-level overview of the new editing application architecture is presented in Figure 4.2. The architecture consists of 4 main elements: 1) NGINX web server, 2) API, 3) data storage, 4) static files. In this section we will discuss each of these components in more detail.

www.pcitvr.nl /assets /api NGINX / Flask API Postgres File System ORM DB queries HTML HTML JS static ﬁles API I/O Data Storage Back-End Front-End

Figure 4.2: High-level overview of the new editing application’s architecture.

4_{https://flask.palletsprojects.com/en/1.1.x/}

(27)

NGINX

The NGINX web server is the central component of the application, and acts as a reverse proxy between the end-user and the application services. It listens to incoming requests from clients, reroutes these requests to the appropriate services, and channels the services’ response back to the end-user. It is also the main entry point when users enter the application, providing the users with the static files that make up the front-end of the web application.

API

The API is the intermediary between the front-end client and the data storage. Its main role is to accept incoming requests, to either store or retrieve data, and to make sure that these requests are handled properly. All communication between the API and database is done via an object relational mapping (ORM). When storing information, the API makes sure that the information is stored in the correct place and in the correct format. On a data retrieval request, the API handles the logic for retrieving the information from the correct place, and providing the client in the format that it expects. The API also makes sure that every request is authenticated and authorized before any data retrieval or storage is performed.

The API uses a RESTful approach, meaning that communication between the client and server is stateless, and all communication is performed through the HTTP protocol. This ap-proach makes the application more scalable, flexible and portable.

Data Storage

As the name suggests, the role of the data storage components is to handle the persistent storage of information related to the PCIT-VR editing application. Two types of data storage are used: PostgreSQL and the local file system. PostgreSQL is used for storing the relational, and mostly numeric and textual data of the application. This data includes profiles of therapists, users, annotations, and more.

One of the requirements of the editing application is that it can store 360° media. Due to

size and efficiency limitations, it is generally considered bad practice to store large files in a PostgreSQL relational database. As such, a secondary storage media is introduced. The file

system is responsible for storing all larger files, such as images and 360° media.

Static Files

The static files are a collection of HTML and JavaScript files that make up the front-end of the editing application. It is considered a separate component of the application, as it is the main way for the end-user to view and edit the state of the application. A deep dive into the architecture for the front-end will be performed in the next section.

4.1.3 The anatomy of the front-end

The component-based approach enforced by React.js causes the general anatomy of the editing application to become a tree-like structure consisting of a single root-node, also called an entry point, and subtrees of child nodes. Each node in this tree represents a component that maintains its own internal state and logic. Each component can additionally have multiple children that it depends on, ultimately creating an hierarchy.

Figure 4.3 presents a subset of the anatomy of the front-end of the editing application. In this figure, we see that there is a single entry component, i.e. the root node, called App. This entity forms the basis for the application, and is the place where all components are combined to become a single application. Ultimately, it is this component that will be rendered to the end-user’s screen. If we go one level deeper, we find the AppRouter. This component has the responsibility to keep the UI in sync with the URL, and contains the logic for rendering the different components based on the URL’s path. The AppRouter component therefore allows us to show different pages for different URLs, regardless of the fact that the application is an SPA. The next layer in the hierarchy consists of components that represent web-pages; isolated views that are linked to certain URLs. An example of this is the root URL /, that is linked

(28)

to Dashboard web-page. The Dashboard is the main entry point of the application, and gives an overview of the projects that a therapist is associated with. Other examples of web-pages are ScenarioEditor, used for editing scenarios, and SceneEditor, the web-page dedicated to editing scenes. What differentiates web-pages from regular components, is that web-pages are mainly an aggregation of multiple components, and handle the logic for communication between these components. The components themselves are smaller, more isolated pieces of code that are assigned a single task, and can be reused throughout the application. These components can be found in the lower levels of the hierarchy. It is important to note that these components can still use other components to extend their functionality.

App

AppRouter

Dashboard ScenarioEditor SceneEditor

ProjectOverview UsersView Analytics

Entry Point .... .... .... .... Pages Components .... ....

Figure 4.3: A subset of the hierarchical anatomy of the application’s front-end

4.1.4 Pages

The editing application contains the following pages listed below. In the sections below, I will further highlight the scenario-editor and timeline-editor pages.

• /projects: Gives an overview of the projects that exist for the therapist – /project/:id: Provides the dashboard for a specific project id • /editor/:id: Provides the scene editor for the specific id

• /scenario-editor/:id: Provides the scenario editor for the specific id • /timeline-editor/:id: Provides the timeline editor for the specific id • /users: Gives an overview of the users associated with the therapist • /analytics: Gives analytics functionality for the therapist

4.1.5 Scenario Editor

Database Schema

The main entity of the scenario database schema, Figure 4.4, is the scenario entity. This entity stores the name of the scenario, its description, and the id of the project that it is associated

(29)

with. A universally unique identifier (UUID) is used as a primary key for the scenario. A UUID is preferred over a regular integer here, because they allow for safe usage in URLs due to less information exposure, and avoid key conflicts between tables. Finally, a reference to the starting scene is added in order to define the root-node of the graph.

Connected to the scenario entity are an array of scene entities. Rather than connecting scenes directly to a scenario, scenes are wrapped in another entity, called scenario scene. Each scenario scene is given its own unique identifier, regardless whether the scene is already connected to the scenario. This allows for reusability of the same scenes by being able to separate duplicated scenes logically.

If we go one level deeper, we find the scenario scene link, the entity that links two scenario scenes together. This entity contains three main elements. The first two elements, the source scenario scene and the target scenario scene, represent the two scenes that the link connects. Due to it being a directed link, a distinction is made between source and target. The third element, action, is a generic entity that is used to represent actions that end-users can perform in the PCIT-VR environment. In the current case, it is simply a wrapper around the option entity, which represents a selectable option of a key interaction moment. I decided to use the generic action entity over connecting the option to a scenario scene link directly, which allows for different types of actions in the future. To elaborate, an action contains meta-information about the action, such as the type, payload and label. This meta meta-information allows us to extend the type of actions that we want to allow in the future, such as interaction with 3D-objects in the virtual space.

1 * 1 * 1 * 1 * 1 * 1 * 1 * 1 * scenario id uuid project_id uuid start_scene uuid

name character varying(128) description character varying(128)

created_at timestamp updated_at timestamp deleted_at timestamp scenario_scene id uuid scenario_id uuid scene_id uuid position_x integer position_y integer scene created_at timestamp updated_at timestamp deleted_at timestamp id uuid user_id uuid project_id uuid

name character varying(128)

description text video_id uuid scenario_scene_link id uuid source_id uuid action_id uuid target_id uuid action id uuid scene_id uuid type type

payload character varying(128)

label text option id uuid annotation_id uuid text text feedback text action_id uuid

Figure 4.4: Database schema design for storing scenarios. Created with https://dbdiagram.io/

API

API endpoints are made available to the client for editing scenarios, corresponding to the re-quirements specified in section 3.1.3. Rather than opting for a single endpoint, I decided to break up the API structure into smaller pieces, allowing each endpoint, listed in Table 4.1, to focus on performing a single task. Though this makes the API structure more complex, it allows for easier development, and more importantly, reduces the payload for each request.

To provide the API with context, the id of the scenario must be provided in the URL, denoted :id. On each incoming request, the API retrieves this parameter, and makes sure that the scenario exists before performing any further actions. If the scenario does not exist, the API will return an HTTP 404 NOT FOUND response back to the client.

A visual editor for creating non-linear timelines for PCIT-VR.

Bachelor Informatica