Remote collaboration in the design process: bridge the physical gap using mixed reality

(1)

Remote collaboration in the design process: bridge the physical gap using

mixed reality

SUBMITTED IN PARTIAL FULFILMENT FOR THE DEGREE OF MASTER

OF SCIENCE

F�� L�

12398160

M�� I�� S��

I�� S��

F�� S��

U�� A��

J�� 4��, 2019

1st_Examiner dr. André Nusselder

Faculty of Science. Informatics Institute

2nd_Examiner dr. Jacobijn Sandberg Faculty of Science. Informatics Institute

(2)

Remote collaboration in the design process: bridge the physical

gap using mixed reality

Fleur Li

University of Amsterdam

Amsterdam, The Netherlands

�eur_memory@live.cn

ABSTRACT

Design thinking is a social activity, where collaborations between multidisciplinary parties are essential. For this reason, remote col-laboration in the design process has been very challenging. Most technological solutions do not address how user behaviors are changed in the context of computer-mediated communication. The current study applied both theoretical and practical methods to gain insights into the process of design and the nature of collaboration. After performing a series of design thinking methods, including interviews, ideation workshop, and a few rounds of iteration, the �nal design "MR war room" was proposed, which uses mixed reality to bridge the physical gap of design collaboration. The exploratory nature of the research ensures both the functionality and emotional meanings of the �nal design. The design solution poses possibilities on how future remote collaboration can be achieved with technol-ogy.

KEYWORDS

Design thinking, Remote collaboration, Mixed reality, Information systems

1 INTRODUCTION

As the world economy shifts from traditional manufacturing to knowledge creation and service production, design has taken a more important role [3]. The goal of design is far more than making things look nicer. As de�ned by Gero, design is a purposeful, constrained, decision making, exploration and learning activity [11]. Business leaders can bene�t from thinking like a designer and integrating design into their business at a large scale [3]. Research showed that design-driven businesses outperform the S&P index by 219% [13]. Companies bene�t from utilizing design thinking in management, decision making, and problem-solving process [13].

As design problems face a great variety of open-ended and com-plex challenges, it requires a community, instead of individuals, to address, frame, and solve these problems [9]. A successful design project is an interplay between completing tasks, collaborating between di�erent parties, and e�cient communications. In other words, design thinking is a shared experience, where communi-cation and collaboration among a group of people are essential. Collaboration refers to a group of people building durable relation-ships and committing to a common goal, and communication is one of the essential elements of collaboration [18]. Moreover, creativity, by nature, is a social activity [9]. New knowledge is often a result of interacting and co-operating with one another [9]. The clashing of more than one idea is the foundation of creating something new.

Taking advantage of shared resources among a group of people, cre-ating ideas in groups can be more e�cient than working alone [29]. However, with the impact of globalization, an increasing amount of companies deal with international clients, and talents are located in di�erent parts of the world. It becomes especially challenging when collaborations are done remotely.

Firstly, the complexity of design problems and its holistic ap-proach requires expert knowledge from di�erent �elds. Designers alone are not su�cient for overseeing all the touch-points of service and the impact of multiple stakeholders [3]. It is crucial to have experts from di�erent backgrounds to bring in di�erent perspec-tives in framing and solving the problem. However, communication within a multidisciplinary team can be challenging. People from di�erent �elds may interpret the same thing di�erently [15]. It is di�cult to check whether multiple parties have a mutual under-standing about thoughts, or whether they understand them in the same way[15].

Secondly, when technology is used for remote collaboration, it changes how people communicate with each other. Researchers argued that through computer-mediated communication, socio-emotional and relational communication is harder to achieve [23]. When the form of media changes, it also changes the way how people think, collaborate, and create [24]. Without su�cient body languages and eye contacts, the context of information and the tone of a text are left open for interpretation. When misunderstanding occurs, it brings anxiety, confusion, and frustration. Productivity, engagement, and innovation are therefore coming at a cost [6]. Moreover, technology has di�erent impacts depending on the type of task that needs to be achieved [17]. The bene�ts gained from the current digital tools are very limited, as they do not take into account the impact of media on how people react, engage, and create.

The current study aimed to understand the nature of collabo-ration in the design process and incorporate them into designing a system that bridges the physical gap. The following research question was proposed:

How might we design a system for a group of physically distributed people to collaborate in a design process remotely and generate inno-vative ideas?

To answer the main research question and set the scope for the current study, a few sub-questions (RQ) are asked:

– RQ1 What are the unique qualities for people who are in-volved in a design development team?

– RQ2 What is the role of collaboration in the process of de-sign?

(3)

– RQ3 What technical and social factors contribute to the success of remote collaboration?

– RQ4 How is creativity achieved in the process of design? The remaining research is structured as follows: literature review, methodology, results, a description of the �nal design, discussion, and conclusion.

2 RELATED LITERATURE

The current section reviews the most recent trends in how technol-ogy is utilized in remote design collaboration. The limitations of current technology solutions are outlined.

2.1 Web-based virtual whiteboard

The Web-based virtual whiteboard is among the most popular so-lutions on the market. Conceptboard1, as an example, provides an online whiteboard where multiple users can sketch, mark up contents, send documents while video chatting with each other at real time. Similarly, GroupMap2_{is designed for brainstorming,} discussion, and decision making speci�cally. Users can post their ideas on a group whiteboard and everyone in the group can review, comment on, or rate the idea. Di�erent ideas can then be sorted and ranked based on the responses from the group members.

2.2 Online collaborative design tools

Other solutions focus more on design tools kits. UXPressia3_{, Smaply}4_, and Touchpoint5_{are all similar tools which help teams visualize,} share, present and improve their customer journey mapping. Simi-larly, Miro6_{is a company creating online collaboration board, which} provides a pre-built template for service blue-printing that teams can directly work on. However, these tools o�er similar features and are limited to very simple tasks. A group of researchers tested the perceived e�ectiveness of an online blueprint editor based on a web application [22]. They found that the collaborative blueprinting tool is useful in case of structuring information and modeling ideas, but users had di�culties toward complex tasks and it results in quitting using the tool. It can be explained by the lack of facilitators who typically explain complex concepts and mediate communication, and this can not be easily achieved by web applications.

2.3 AR, MR, and VR

More recently, the boundaries between the physical world and computer-generated virtual world are broken down by augmented reality (AR), mixed reality (MR), and virtual reality (VR). They revolutionized how people communicate with each other and are therefore used in many aspects of design collaboration.

1_{https://conceptboard.com/} 2_{https://www.groupmap.com/} 3_{https://uxpressia.com/} 4_{https://www.smaply.com/} 5_{https://touchpointdashboard.com/} 6_{https://realtimeboard.com}

To understand how 3D visualization technologies are utilized in remote collaboration, it is important to understand the di�erence between them (Figure 1). AR, MR, and VR can be seen as a con-tinuum of reality. Reality is what takes place in the physical and material world, and can be perceived by us without the help of IT [8]. AR is an additional layer of digitally created images on top of the reality. MR creates digital contents that are able to interact with the actual world in real time. VR, on the other hand, is a complete, 3D digital representation of the actual world.

Figure 1: The di�erence between AR, MR, and VR is illus-trated. AR is simply an overlay of virtual images on a phys-ical environment; MR enables the physphys-ical and virtual con-tents to interact with each other, which is illustrated by the bear hidden behind the chair; VR is completely virtual, and the physical environment is blocked out.

2.4 Collaborative environment with 3D

visualizations

With the advancement of sensors and computer vision, collabo-ration within 3D environments is possible. Scientists combined smartphones, Microsoft Kinect, and multiple depth sensors to cre-ate 3D mobile collaborations [10]. Two people, a local user, and a remote user, can see each other’s hands in the form of 3D avatars. Re-searchers also managed to scan the complete physical environment and reconstruct it into textured 3D geometry. Tait and Billinghurst were the �rst to develop a system using depth scanner to achieve view independence [25]. The complete workplace of the local user is �rst scanned by a depth camera, then the remote user is able to see the environment through a desktop interface. The remote user can navigate the environment independent of the local user’s point of view. View Independence is very important in remote collabora-tion as it increases the e�ciency and reduces the time spent on a single task [25].

There are a few products available on the market, which com-bine the latest technologies mentioned above to achieve design collaboration. Prism7_{and Spacial}8_{are two similar mixed reality} platforms that enable remote collaboration through real scale body avatars. With the use of Microsoft Hololens, local users can see and manipulate 3D objects side by side with team members who are represented by their customized avatars. Prism also supports visual communication with features like virtual laser pointers and 3D line drawings. Users can upload real scale 3D models and interact with

7_{https://objecttheory.com/platform/} 8_{https://spatial.is/}

(4)

them through Hololens. In this case, collaborative mixed reality provides a good tool for interactive content creation, product de-sign, 3D modeling, and optimization. However, those platforms on the market are not tested scienti�cally. It is hard to know how well they actually perform, what are their limitations, and whether they do have a positive impact on remote collaboration.

3 METHODOLOGY

To answer the main research question and sub-questions, �ndings were gathered from two perspectives: theoretical and practical. Theoretical �ndings were generated from literature study, which aim to answer RQ3, the technical and social requirements for remote collaboration in the design process. Practical �ndings were gathered from a series of design thinking processes, which aim to answer all of the research questions.

The design thinking processes started with one round of inter-views. The interviews aim to gain insights into the needs and behav-iors of the users, as well as validating the theoretical �ndings from the literature. Next, a set of requirements were generated, based on the results of both the literature study and interviews. Then, an idea generation workshop was organized among six experts from KPMG. Based on the initial ideas created in the workshop, a prototype was created and tested. Finally, the �ndings from proto-type testing were integrated into the �nal design. In addition, the current study aims to understand the process of design, by using the design thinking methodology. When the keyword "design" is mentioned, it is important to distinguish the di�erence between the content of the research, and the methodology it uses.

3.1 Literature study

A literature study was performed to approach the research ques-tions from a theoretical perspective. Multidisciplinary Research from cognitive psychology, communication, and computer-human interactions was gathered and analyzed to answer RQ3. The results were divided into multiple sub-sections as a few important factors to consider when designing remote collaboration tools.

3.2 The design thinking method

Design thinking methods approach the research questions from a practical point of view. It translates observations on humans into insights, then further into designing solutions [3]. In the current study, there were many people-involved activities. Insights were gained from interviews, an ideation workshop, prototyping, and testings, which contributed to the decision makings later in the �nal design.

3.2.1 Interviews.

One round of semi-structured interview was conducted with a total of 9 participants. All of the participants were recruited from within the company of KPMG. It included 5 designers (D1 to D5) and 4 non-designers (N1 to N4). The designers were experts in using de-sign thinking to solve a variety of complex business problems. The non-designers have experiences overseeing design processes and have worked with designers before. This group consisted of a scrum

master, a process designer, a senior manager, and a product man-ager. To understand the process of design and how collaboration is achieved, it is essential to consider the needs of both designers and other parties involved. Design collaboration usually involves a multi-disciplinary group of people [3]. Therefore, the interview aims to gain insights on the same process, but from two di�erent perspectives.

One set of interview questions was developed to guide the pro-cess. Each of the interviews was conducted face to face in a private conference room, and questions were asked based on the interview guide (Appendix A). Each interview lasted 30 to 50 minutes and the conversations were recorded for analysis. The nine interviews were �rst transcribed and then coded based on the research ques-tions. The computer software QDA Lite9_{was used for coding the} interviews. Multiple codes were �rst created, and similar codes are grouped into four categories based on the research questions. Each category is developed based on each sub-question of the current study. The detailed coding scheme can be found in appendix B.

3.2.2 Ideation workshop.

An ideation workshop was organized among potential users. Six experts from various backgrounds were invited to the workshop session, including two designers, one product manager, and three developers, all of whom have experiences working in a design pro-cess, and are involved in some forms of remote collaborations. It was hosted in a living-room style conference room, with comfy sofas and co�ee tables. Pictures of the workshop day are attached in appendix C. The casual setting of the physical location aims to create a safe, relaxed, and open environment for everyone to work together. During the 2 hours long session, �ndings from the inter-views were �rst presented by the researcher, then a few interactive activities were included to investigate human needs, generate ideas, and rate ideas. Finally, the top voted idea was made into a prototype with Lego, which served as a foundation of the �nal design.

3.2.3 Prototyping and testing.

To verify the initial design idea, a lean prototype was created and tested. It was adapted from the Lego prototype created in the work-shop, with additional design decisions made based on the �ndings from the literature and interviews. This initial prototype was then tested among 8 potential users. Testing a prototype early in a de-sign process allows rapid resolution of dede-sign errors and validates whether the design meets the needs of the users [31]. Testing was done one-on-one in a private conference room. A few questions were asked based on the experience of the participants.

4 RESULTS

4.1 Literature study

To answer RQ3, the social and technical requirements of remote collaboration, a literature study was performed. It included studies that investigate the behaviors of people in the context of using re-mote collaboration technologies. The most important requirements for achieving e�ective collaborations are explained in details in the following sections.

(5)

4.1.1 Cognitive and functional seams.

The use of technology often sets barriers in communication, which takes up extra cognitive e�ort. In most computer-supported col-laboration interfaces, there are two kinds of seams, or barriers, that a�ect the e�ciency and experiences of task performances: functional seam and cognitive seam. Functional seam refers to the gap between di�erent functional workspaces, switching between which disrupts focus and decreases task e�ciency. It forces the user to change the mode of operation, such that the user cannot have e�cient eye contacts while typing notes another window [2]. Cognitive seam refers to the disconnection between existing tools and new ways of working [2]. For example, letting an oil painter use a digital painting tool will force them to adopt new ways of working. The learning curves and the extra cognitive e�ort are the results of cognitive seams.

4.1.2 Tangible interaction.

Tangible interaction is crutial for collaborating remotely. It not only aids communication but also encourages creativity. Lee and colleagues argued that exploration through physical actions can improve design cognition [15]. Compared to the traditional graphi-cal interface using mouse and keyboard, interacting with graspable objects takes o� the cognitive loads on spatial reasoning. It en-ables users to interact with task objects the same way as in the real world, therefore provides a�ordances to virtual objects. Moreover, research showed that gestures are not only related to communica-tion, but also to thinking [12]. When thinking is facilitated by accommunica-tion, the thoughts bounce back and alter behavior. The feedback loop between thinking and action encourages learning and creativity.

4.1.3 Spatial faithfulness and trust.

Having realistic spatial relationships between di�erent people is important for remote collaboration. As de�ned by Nguyen and colleagues, spatial faithfulness refers to the extent to which the spatial relationships between participants and the environment are preserved [19]. With the use of traditional desktop-based video conference, the spatial cues and eye gaze directions are lost. As a result, it requires more cognitive e�ort for people to map the spatial relationships in their mind on top of maintaining an conversation [28]. Wang and colleagues proposed a system to achieve eye gaze awareness [28]. They put separate computer screens at di�erent spatial angles in respect to the participant, and cameras attached to each of the screens were used to capture the direction of gaze [28]. Each incremental step in spatial faithfulness supports the formation of trust, which in turn results in the resilience in cooperation and decreases negative factors such as fear, awkwardness, and ambi-guity [19]. An example of full spatial faithfulness is face to face (F2F) communication or an equivalent system that achieves the full capability of F2F communication [28].

4.1.4 Visual information and co-presence.

Siegel and colleague argued that visual information is used as a con-versational source in the accomplishment of tasks [14]. A smooth and natural communication is based on the common ground be-tween communicators. Common ground refers to the shared knowl-edge, beliefs, and assumptions which help communicators antici-pate what the partner knows and thinks [4]. Clark and Marshall

identi�ed three sources where common ground can be created [4]. First, people might have common ground prior to the interaction, when they are from the same group or community. Second, common ground can be achieved with linguistic co-presence, as when parties in the conversation use pronouns, such as you, me, and they, to refer to a previously mentioned object or concept. Third, common ground can be built by physical co-presence, where communica-tors are physically present in the same environment. Co-presence provides a rich set of resources, such as sights, smell, and touch sen-sations, which conversations can be built upon. Visual information is one of the most important resources, especially within the con-text of computer-mediated communication [14]. Among di�erent types of visual information, facial expressions and gestures can be used to determine the level of comprehension and the general area of attention. Workspace environment can set the scope of what is most likely to be talked about and help partners make sense of each other’s action. When communicators are remotely distributed, visual information can help di�erent parties to see the task objects, the environment, and each other’s behaviors.

4.1.5 Summary of requirements (R) based on literature. • R1 Users are able to shift between multiple task windows

seamlessly

• R2 Users are able to intuitively adopt task interactions with virtual objects, based on how these interactions are achieved in real life

• R3 Users can use gesture or "touch" to interact with tangible objects, even if the objects are virtual

• R4 Spatial relationships between people, between people and environments, should be preserved

• R5 Visual information should be shared as much as possible between physically distributed users

4.2 Interview �ndings

The interviews aimed to gain a deeper understanding of the design process as well as the needs of the users. The results are structured as follows: the user persona was �rst created to illustrate the unique quality of people who are involved in the design process (RQ1). Then, the importance of collaboration in the design process was outlined (RQ2). Next, the requirements of the users were identi�ed (RQ3). Last, how creativity is achieved in the process of design was explained (RQ4).

4.2.1 RQ1: The unique qualities for people who are involved in the design process.

To understand the needs, behaviors, and motivations of the users, a persona was created (Figure 2). Mia is a 35 years old innovation consultant. Her daily responsibility is to use design thinking to solve complex business problems. Although her main role is a de-signer, the actual tasks of visual design, ideation, and wire-framing are only one part of her responsibilities. At the same time, she acts as an organizer and facilitator within a design project. She organizes meetings, talks with end-users, brings the right people to the workshops. Sometimes, she is not the one who is generating ideas. Instead, she facilitates workshops and sets up an environment

(6)

where other people, such as clients and stakeholders, are comfort-able to participate in the idea generation process. One of her biggest motivations is to inspire people from di�erent backgrounds to cre-ate unexpected good ideas. When she facilitcre-ates workshops, she takes up a "back-end" role to guide people throughout the chal-lenge. She uses her skills and experiences to encourage people to be creative in his or her own ways.

Throughout the entire design process, it is crucial to collaborate with multiple parties, such as end-users, product owners, stakehold-ers, and clients. She has intensive interactions with them in various settings. Meetings and workshops are common working environ-ments for Mia. Phone calls and online collaborations are necessary when the other party is at a di�erent location. In general, she needs a safe environment that supports e�ective communication, where trust can be built and people feel comfortable bringing in ideas.

Figure 2: The user persona was created based on the views. It includes the job responsibilities, likes, needs, inter-actions, and work settings of the users.

4.2.2 RQ2: The importance of collaboration in design processes. To gain a deeper understanding of the design process and the role of collaboration, the designers were asked to sketch a past design process on a piece of paper and include important interactions (Appendix D). This exercise was designed to be at the beginning of the interview, which helps people to revisit a past experience and recall the details of that experience. It was an important step toward understanding the form, nature, and importance of collaboration and how it is embedded in a design process.

The �ow chart at the top of �gure 3 is a simpli�ed version of the sketches, which can be seen as a typical design process. Five steps were included: de�ning problems, research, workshop day, proto-type & validation, and development. Vertically, three properties within each step are listed: importance level, parties involved, and requirements. Each dot represents one interviewee, such that a blue dot represents a designer (D1 to D5) and a purple dot represents a non-designer (N1 to N4).

Five out of 9 people identi�ed the prototyping & validation phase to be one of the most important steps (some of them picked more than one important steps, Figure 3). It is important to show

prototypes to other parties as early as possible to get feedback. To make sure the design ful�lls the needs of the users, having many rounds of iteration and testing is crucial. D4 mentioned that "If you go from design to products as fast as possible, you can best adjust because you will get real feedback once it’s alive or produced." D3 described the prototyping & validation process as following: " So you prototype then you test. You put the �ndings back into the prototype, and so you might do it for X number of times."

The second most important step is to de�ne the problems. The aim of the design process is to solve a problem, which usually in-volves many stakeholders and is embedded in a complex situation. Identifying the real problem can be challenging. Clients usually have a pre-existed way of what problem should be tackled, but some-times there is a lot under the surface of that problem. D5 compared the design problem with an iceberg: "you know like the iceberg, when you just ask them questions, people will tell you this bit (tip of the iceberg), but there is a lot (under the surface). " Moreover, the problem de�nition phase determines the direction for the rest of the design process. Collaboration and e�cient communication are the key to a well-de�ned problem. N3 gave an example of how to dig deeper to approach the real problem: "You want to understand the problem as much as possible. We call it ’three levels of why’. If you ask why enough times, you get to the core of the problem."

Overall, every stage of the design process involves many dif-ferent parties (Figure 3). Participants stressed the huge impact of collaboration on the design processes. D4 mentioned that "I very strongly believe you cannot design on your own. I mean you can, but you can never make anything good." D5 explained why working with others is important: "you really need people to challenge you, to validate things with, or to get you a bit more out of your comfort zone".

Figure 3: A summary of the interview �ndings. People’s opinion on the important steps in the design process, di�er-ent parties involved, and the requiremdi�er-ent are mapped under each step within a design process. The blue dots represent designers and the purple dots represent non-designers.

(7)

The users identi�ed many requirements that ensure the success of collaboration in the design process (Figure 3). The requirements mainly focus on the workshop day, as it is usually a group activity and hard to manage in a remote setting, but those requirements generally apply to many other design stages as well. Among all the requirements, having shared task objects and face to face interaction are the most important factors in design collaboration.

Shared task objects.

The workshop brings many important parties and stakeholders together to form a clear vision and create ideas. In this step, the designer acts as a facilitator. They guide people throughout the process of ideation. The participants at the workshop, often times clients and end-users, will come up with ideas themselves. Because they have created the ideas themselves, it creates a sense of owner-ship and the ideas are more likely to be adopted.

There are some unique characteristics of a design workshop, which make it di�erent from any other forms of collaboration. The use of shared task objects is a major requirement for a design workshop. A shared task object is something that everyone in the group can see, refer to, or make changes on. A design workshop usually involves three kinds of shared task objects: physical tools, demos, and creativity enhancing objects. Physical tools are used to facilitate the activities within a workshop, such as whiteboards, post-it notes, pens, and paper. Demos can be a prototype or a piece of design. Creativity enhancing objects can be toys, puzzles, or Lego, which designers use to discover the creativity of the participants. Without these physical objects, design collaboration can be really di�cult. It is one of the biggest frustration of the users. "I think it can be frustrating when you cannot work together on one particular digital thing at the same time. Those kinds of things you can’t work together (D2)."

Face to face interaction.

Many users think design workshops cannot be done remotely be-cause there are not su�cient communication and feedback. Eye contacts and facial expressions contain plenty of subtle social cues that help people understand each other. N1 mentioned that: "If you can see someone on the other side, that’s essential. Quite often, they say they understand it, but they understand half of it." Having a back and forth discussion is how ideas are created. "I found that (calling on the phone) is impossible because you can’t work out whether they have understood what’s going on. There’s a boundary in some sense, for me personally (D2)."

4.2.4 RQ4: Creating innovative ideas in the design process. The interview results con�rmed the literature �nding (section 4.1.2) that tangible interactions and visual information can support idea generation. D5 mentioned that "I think visual (displays), language, and making drawings together are really important. It really helps to get your other part of the brain active and come up with di�erent ideas." Moreover, working with others can be better than generating ideas alone. D3 mentioned that "The best way I can be creative is to always work with at least one partner. You need to bounce ideas o� of people, because expressing ideas sometimes sparks something else".

4.2.5 Summary of requirements based on interview findings.

• R6 Having shared task objects that everyone can interact with at the same time

• R7 Being able to see each other’s facial expression and make eye contacts

• R8 Providing an easy way to show demos and prototypes, and people can easily leave comments or make changes

4.3 Ideation workshop

In order to generate innovative ideas based on the requirements, an ideation workshop was organized internally at KPMG. The total length of the workshop was two hours. It started with the researcher presenting the �ndings from the interviews and followed by a few group activities of de�ning human needs, ideation, and prototyping.

4.3.1 Defining human needs.

De�ning human needs is an important step to understand the value and meaning of the current design problem. Although it is clear that we want to solve the problem of remote collaboration, the underly-ing meanunderly-ings that drive this goal are unclear. The exercise "levels of insights into human needs" [20] was used to help participants reframe the current research question and gain a deeper under-standing from the perspective of humans. Re-de�ning the meaning behind problems can often lead to radical innovation, where new and unique ideas are created [26].

The participants completed this exercise in groups of three, and two groups arrived at slightly di�erent values:

– Group 1: Future interactions should create a safe space for communication in order to encourage creativity.

– Group 2: Future interactions should empower people to ex-press their ideas as e�ciently as in person.

Both of these meanings are valuable and served as a guideline later in the ideation phase.

4.3.2 Ideation.

After identifying the meanings and values behind the problem, each participant was asked to come up with ideas within a short amount of time. The method "crazy 8"10_{was used. It is a sketch exercise that} challenges people to come up with 8 distinct ideas in 8 minutes. The aim was to diverge people’s thinking and generate a variety of ideas without worrying about the details and viability. After 8 minutes, everyone presented their ideas quickly in front of the whole group. Then, participants were asked to discuss with each other and group similar ideas into categories. The next stage was then to vote for the top three ideas, based on the usability, scalability, and feasibility of the ideas. Each participant had three yellow stickers and can put them next to the categories of ideas to vote. The grouping of ideas and the result of the voting are illustrated in �gure 4. There were six groups of ideas in total. Each group and the number of votes are listed as following:

– The virtual "War Room" (5 votes): a virtual room where everyone can collaborate together

– The "project time" (4 votes): a standard time reference for people working at di�erent time zones

– The "Who moved my cheese" (2 votes): version management in virtual space

10_{https://designsprintkit.withgoogle.com/methodology/phase3-sketch/crazy-eights} 6

(8)

– The "Send me there" (1 vote): make a digital copy of people and let them work together in virtual space

– Online collaborative whiteboard (1 vote)

– The "Smart o�ce" (1 vote): using a smart board and smart wristbands to draw together

Figure 4: The ideas created in the "crazy 8" exercise were grouped based on similarities. The most voted idea was "the virtual war room", which had 5 votes, and the second most voted idea was to create "project time", which had 4 votes.

The top voted idea used the concept of the war room. A "war room", in the context of design, refers to a centralized workspace with plenty of visualization tools, such as whiteboards, diagrams, and designs. It helps people to keep an overview of the project as well as making important decisions [27]. The idea was that even when people are far apart, they can enter the virtual war room through the use of AR or VR glasses. In this immersive environment, users can interact with each other through gestures, drawings, and facial expressions as if they are side-by-side with each other.

4.3.3 Prototyping with Lego.

In the last stage of the workshop, the "virtual war room" idea was made into a prototype with Lego. Lego was chosen as the proto-typing method because the hands-on experience of manipulating physical objects can help people further re�ne their ideas and stim-ulate new thinking [16]. To present a design idea e�ciently, partici-pants were asked to focus on four key components: people, objects, location, and interactions [5]. The outcome of the prototype is il-lustrated in �gure 5. The green plate represents the physical world, while the white plate represents the virtual environment. In the physical world, people are located in di�erent places in the world. With the use of AR or VR glasses, they are able to work with each other together in the virtual room. The virtual room consists of computer-simulated 3D objects, automobile parts in this case, and each group member was represented as 3D avatars, in the virtual space. They were able to move things around and talk to each other as if they are physically present in the same room.

4.4 Prototyping and testing

Based on the top voted idea from the workshop (section 4.3.3), the concept, "virtual war room", was taken as a foundation for the

Figure 5: The end product of prototyping with Lego. The left image represents people in the real worlds, and the right im-age is the virtual room where they can work together with each other with the help of AR or VR glasses.

current design. To simulate the "look and feel" of the interface and verify the initial design idea, a lean prototype "MR war room" was created. It uses Microsoft HoloLens11, the mixed reality headset, to simulate the environment of remote collaboration. With the use of HoloLens, the physical environment of the user and virtual objects are merged together to facilitate communication and collaboration. This section starts with an explanation of why mixed reality was chosen as a media for remote collaboration. Then, the prototype is presented. Last, the �ndings from prototype testing were outlined as additional requirements that needed to be Incorporated into the �nal design.

4.4.1 Why mixed reality.

Billinghurst and colleague argued that mixed reality is ideal for collaborative work[2]. MR provides opportunities to minimize func-tional and cognitive seams. As mentioned in section 2.3, the virtual and physical contents within the MR interface can interact and are "aware" of each other. This allows users to work with virtual con-tent within their own physical space, where the workspaces are not limited by the size of a screen. Moreover, integrating virtual objects to the physical environment provides a�ordances they can interact with. When the objects are presented in 3D forms, it is easier for users to know what to do with it and take possible actions.

4.4.2 Prototyping.

Figure 6 captured the live demo of the prototype. A conference room was used as the war room, where the remote user (RU) can join and collaborate.

In this room, there was a whiteboard with a hand-drawn diagram. Next to the whiteboard, the RU was displayed as a life-size video call window. The video call was imitated by a 1.5 minutes long pre-recorded video. In the video, the RU communicated with the local user (LU) on the upcoming project and discussed the diagram on the whiteboard. As shown in �gure 6, the RU maintained eye contacts and used body gestures to actively engage the LU in the conversation. To illustrate her thoughts better, she used spatial annotation to make changes to the diagram on the whiteboard. Figure 7 demonstrated how spatial annotation was made. The RU is able to overwrite, circle, or draw on top of the hand-drawn diagram.

11

(9)

Figure 6: Both images are the screen capture of the live pre-view of the prototype. The left image shows a user persona pinned on the left side from the perspective of the user. The right image shows the �eld of vision in the front.

The changes made by the RU are displayed as a virtual layer on top of the physical environment. Moreover, the RU is able to send over digital contents and pin them in the LU’s environment. In this prototype, the RU sent over a user persona and pinned it on the left wall. A few pauses were added throughout the video to give the LU a feeling of actually engaging in a conversation. Although the video was pre-recorded, the life size display and the body gestures are aimed to give a realistic feeling of co-presence.

Figure 7: A demonstration of spatial annotation. The RU is able to draw, circle, and overwrite in the LU’s space. It is dis-played as a virtual layer on top of the physical world.

4.4.3 Testing.

This initial prototype was then tested among 8 potential users. For the current design of MR war room, the goals of prototype testing are two folds:

– to understand how the "MR war room" environment a�ects the ease of communication within the context of design development

– to get open feedback from the users on what needs to be improved for the future design

Each participant was �rst given an introduction of the scenario and the overview of the testing procedure. Su�cient details were given to simulate a realistic work-place situation. They were told that their colleague cannot be here in person and had to collaborate remotely. They were �rst assisted to put on the HoloLens. Once the video started playing, the researcher remained neutral, and the participants were able to experience the interface themselves.

Afterward, the participants removed the headset and were asked a few questions based on their experience.

Overall, the participants reacted positively to the prototype. The MR environment made the collaboration more engaging and tuitive, as the video was accompanied by simultaneous visual in-formation. One of the participants commented that visualization helped him understand the tasks better. It would be potentially even more helpful with a more complex info-graphic or �owchart. Another participant appreciated that the changes made by the RU can be seen instantly, and the changes he would make can be seen by the RU as well. This kind of interaction makes collaboration easier.

Being able to refer to the common task object provides an op-portunity to bring up di�cult conversations. Most participants feel comfortable to express a disagreement because there are fewer risks of misunderstanding. "This technology solution would not put any boundaries on that (engaging in di�cult conversations), it might actually pose an opportunity because it makes it more clearly than just over a call and etc.". However, it is based on an assumption that the RU can see the user as well as they see the RU: "If I know he (the RU) can see me as well as I can see him, I would be more comfortable (with raising di�erent topics)".

There were a few things the participants thought can be im-proved. The majority of them indicated that more interactions would make the experience more intuitive. "If it becomes more interactive, that you can actually see someone doing something... it becomes more real." Four participants argued they would like to see RU in the form of 3D avatars, to make use of the 3D visual-ization technology of the HoloLens. They thought the video chat window was an advanced version of Skype. The future prototype can present the RU in 3D format and let them walk around the room with them. In addition, being able to draw together is a crucial user need. One participant pictured what ideal interactions look like: "maybe we are both writing on the whiteboard at the same time... or maybe we grab a piece of paper and do something here on the �at surface". Lastly, the progress of the drawing and annotations should be saved, so di�erent rooms can be used by di�erent projects. Users can re-enter the war room every time they collaborate on the project.

4.4.4 Summary of requirements based on the findings.

Based on the �ndings of the prototype testing, additional require-ments were outlined.

• R9 The ease of communication should go both ways. The users on both sides need to be con�dent that the other user can see and hear them well.

• R10 The users should be able to move around in the room. • R11 The presence of the RU should be more realistic than a

video window preferably in the form of 3D avatars and able to move around in the room.

• R12 There should be more interactions. Being able to draw together and see what each other is pointing at are very important.

• R13 The system needs to remember the progress of the project and the virtual information of the room should be saved.

(10)

5 THE FINAL DESIGN: THE MR WAR ROOM

The current section presents the �nal design based on the require-ments generated from the literature study (R1 to R5), the interviews (R6 to R8), and the prototype testing (R9 to R12). The MR war room allows users to collaborate remotely as if they are physically present in the same room. It is an iteration of the previous prototypes and a product resulted from many design decisions. The current section starts with a scenario, where the MR war room is used in a real-life situation. The requirements (R1 to R12) are incorporated into the description to illustrate how each design decision was made. Subse-quently, di�erent aspects of the interaction are described in details, followed by a short explanation of the supporting technology. Al-though the full technical implementation is beyond the scope of the study, the short technological explanations are aimed to provide a holistic view of the �nal design.

5.1 Scenario

Tom is an innovation consultant in a large company. His job requires intense collaboration with multiple parties and stakeholders. For the upcoming engagement, he works with a traditional hotel chain to innovate their services. Throughout the process, he needs to collaborate with Julia, the representative from his client company. However, Julia locates in the U.S., and needs to collaborate with Tom remotely for most of the time. He has access to HoloLens at work, so he wants to give the "MR war room" a try. It is a HoloLens app that enables remote collaboration in the design process. In his o�ce, he hangs visual materials relevant to his projects on the wall. He wants to refer to this information while talking to Julia.

Once he sets up the HoloLens and launches the "MR war room", he is asked to �rst scan all surfaces of his environment. Then, within the interface, he uses the call feature to connect with Julia. Once Julia is connected, she is presented as a life-size 3D hologram (R11). The hologram of Julia is co-present with Tom and is able to walk around in his o�ce (R4, R10, R11). They can see each other’s facial expression, gestures, and what they are working on (R1, R7, R9). On the other end, Julia is also wearing a HoloLens and she is able to see Tom as a hologram (R5, R6, R9). Tom wants to show Julia a sketch he drew on his whiteboard (R6, R8). They gather around the whiteboard (R10) and make changes as if they are co-present in the same room (R1, R2, R4, R5, R12). While explaining her idea, Julia uses gestures to draw and annotate the sketch (R1, R2, R3, R5). Tom uses a physical marker to draw on the whiteboard to illustrate his idea (R12). The drawing he made can be seen by Julia instantly (R12). After the meeting, when Tom takes o� the HoloLens, the virtual annotations disappear, but the physical environment stays. Next time when they want to collaborate together, they can re-enter this room and the annotations will be preserved (R13).

5.2 Interaction design

5.2.1 The MR war room environment.

As section 4.3.2 brie�y mentioned, designers use war-room as a centralized workspace where all the necessary visual information is available at a glance. With the use of Microsoft HoloLens, the "MR war room" allows RU to "enter" the war room and work together with the LU (Figure 8). It serves as a canvas for shared note taking and long-term storage of work in progress.

Figure 8: The interface of the MR war room. The woman at the right edge of the image is the hologram of the RU. The man in the picture is the LU. The large graph on the left side and the phone screens in the middle represent the virtual objects that can be placed in the war room. The circles, ar-rows, and letters on top of the whiteboard represent spatial annotation.

To construct a shared working space, the war room is duplicated and projected into the RU’s environment in real scale. The LU needs to �rst use the HoloLens to scan the surfaces of the war room. It includes turning around for 360 degrees and gazing shortly at all surfaces of the room. The war room, including the visual displays, is scanned and rendered into a 3D model in RU’s environment. When the RU joins the session, his or her hologram appears in the LU’s environment. On the other side, the RU can see the 3D model of the war room, as well as the LU’s hologram. When the user on one side moves around, his or her hologram moves around on the other side. Users on both sides experience the same space, but with a di�erent combination of physical and virtual objects (Figure 9).

Figure 9: The table shows how the same visual information is presented in both the LU and RU’s environment. The user always sees the other user as virtual holograms. The added virtual objects and spatial annotation are also virtual for both users. The war room and the visual displays within it are real for the LU and virtual for the RU.

Virtual objects such as post-it notes, sketches, or 3D models can be added into the war room. Because virtual objects are aware of and can interact with the physical world, users can put virtual objects into the physical world, such as pinning a design sketch on the wall, placing a virtual post-it note next to a real one, or laying a diagram on the surface of a table. To move these virtual objects around, users need to �rst have their index �nger pointing to the

(11)

roof, then tap, hold, drag, then release12_{. With the added layer of} virtual elements, the MR war room turns into a long term memory of both physical and virtual objects, that can later be accessed at any point.

Spatial mapping.

Spatial mapping enables virtual objects to interact with the physical environment in a convincing manner. The depth camera within HoloLens scans the surface of the environment, and a mesh overlay-ing the environment is created [1]. The HoloLens constantly scans the room when the user navigates within the environment, and the mesh is very frequently updated. The mapping data of the room and the virtual objects within it are stored in the local database. The data is linked with Wi-Fi characteristics, so one than one room can be mapped and stored. Every time the user enters a location, HoloLens identi�es the space and retrieves the mapping data auto-matically, which enables the "MR war room" to store information for a longer time.

Room scanner.

The four cameras and one depth sensor within HoloLens enable users to map the geometry of the environment while tracking the indoor locations of the device. Applications such as HoloLens Room Scanner13_{, are able to capture a full colored 3D model of any room.} This information can be uploaded, shared, or displayed in another physical environment. This enables the "MR war room" to copy the environment of the LU and place it as a real scale in the RU’s environment. Since the indoor location of the device can be tracked, not only the indoor environment but also the user’s movement within the environment can be simultaneously captured [30].

5.2.2 Co-presence with another person’s hologram.

With the RU presented as 3D holograms, it allows more interactions, such as walking around and draw together. Figure 8 illustrates the user interface of the LU. The woman at the right edge of the image is the hologram of the RU. The guy in the picture, the LU, in this case, can see her as real scale and communicate with her as if they are in the same room. Because the indoor location can be tracked, the hologram of the RU can simultaneously move around in the environment. This is true for the LU as well.

Holoportation.

Co-presence can be achieved with holoportation [21]. Microsoft research developed a system to achieve virtual 3D teleportation of people in real-time. LU and RU can see, hear, and interact with each other in 3D holograms [21]. In order to capture the 360 degrees of the subject, 8 camera pods are set on the periphery of the room, pointing inward to capture a unique angle of the subject. Real-time depth estimation techniques combined with masking of the background allow only the regions of interest to be reconstructed. Then, texture and color are rendered, and the spatial sound stream is added. The data then can be compressed and rendered in a remote device [21]. To capture the eye contact, two wireless video cameras can be placed between the eyes of the users and the device to create an augmentation of the user’s eye gaze [21].

12_{https://docs.microsoft.com/en-us/windows/mixed-reality/gestures} 13_{https://www.matrixinception.com/wp/products/hololens-room-scanner/}

5.2.3 Spatial annotation.

Spatial annotation allows users to make notations on real-life ob-jects. In �gure 8, the spatial annotation is illustrated by the circles and arrows sketched on the whiteboard (located at the far back in the image). To make annotations, the user can simply use the index �nger to draw in space. A cursor will appear in the user’s �eld of vision to indicate where in space the drawing lands on. The user can move the arm forward and backward to determine how far away in space they want to draw.

6 DISCUSSION AND FUTURE WORK

The current study aims to understand the nature of collaboration in the design process. Based on the �ndings, a design of a remote collaboration system was proposed at the end. The study con�rmed the assumption that collaboration is at the core of design processes. As shown in �gure 3, people from di�erent backgrounds and with di�erent job responsibilities are gathered to solve design problems at each stage. Design collaboration di�ers from other forms of collaboration because it relies heavily upon shared task objects and requires a rich form of communication.

Past research focused on the technical enablement on one aspect of remote collaboration, such as mutual awareness [28], tangible interaction [7], and spatial faithfulness [19]. The current study took one step back and investigated patterns and meanings rooted in the behaviors of users. This human-centered approach integrated �ndings from multiple �elds. It, therefore, ensures both the func-tionality and the emotional meanings of the �nal solution. The design thinking methods in the current study supported the ex-ploration in possible design solutions. Being exploratory instead of conclusive provides opportunities to understand the problem within its bigger context.

While the current study is rooted in theory, it also has strong practical relevance. Various steps in the research process involved collaboration with di�erent parties. Speci�cally, the �nal design of "MR war room" was based on the collective e�ort at the ideation workshop, not the creation of a single individual. The involvement of a multidisciplinary group of people provides alternative visions in the process from synthesizing �ndings to designing a solution. Only one round of iteration was done in the current design. Ideally, a few more rounds of testing should be done to gain more understanding of how remote collaboration is addressed in the designed solution. Moreover, some quantitative methods can be used to gain deeper insights into the "MR war room". It will be interesting to see how well the LU perceives the RU as co-present with them, does it perform better than a video call, and whether it feels as realistic as physical co-presence.

Although the exploratory design method has its bene�t of in-tegrating various research into a single design, there are some technical and usability limitations that need to be further validated in future work:

– Whether the image quality of the 3D holoportation is good enough to provide a good user experience

– What is the minimum physical space required to collaborate within an MR environment

(12)

– How to achieve group collaboration, where three or more people are involved in design collaboration

– How is the user experience of wearing the head-mounted device

The brief description of the technical principle behind each in-teraction (section 5.2) provides evidence that the above technical limitations are possible to be addressed in the future. If the MR war room can provide good usability and functions at a relatively high level, this remote collaboration tool can be adopted to other �elds of remote collaboration. Even the simplest form of collaboration would bene�t from a rich communication and intuitive interactions.

7 CONCLUSION

The current study combined theoretical and practical methods to understand the nature of collaboration in the process of design. A design solution addressing the social and technical needs was proposed. It poses a possibility of how technology can be used to bridge the physical gap of remote collaboration and points to the future research direction of achieving more e�cient design collaboration with technology.

REFERENCES

[1] Kelly Baker. 2018. Spatial mapping - Mixed Reality | Microsoft Docs. (2018). https://docs.microsoft.com/en-us/windows/mixed-reality/spatial-mapping [2] Mark Billinghurst and Hirokazu Kato. 2002. Collaborative Mix reality. Commun.

ACM 45, 7 (7 2002). https://doi.org/10.1145/514236.514265

[3] Tim Brown and Barry Katz. 2011. Change by design. Journal of Product Innovation Management 28, 3 (5 2011), 381–383. https://doi.org/10.1111/j.1540-5885.2011. 00806.x

[4] Herbert H. Clark and Catherine R. Marshall. 1981. De�nite reference and mutual knowledge. Elements of discourse understanding (1981), 10–63. https://doi.org/10. 2749/101686611X13131377725361

[5] Rikke Dam and Teo Siang. 2018. Prototyping: Learn Eight Common Methods and Best Practices. (2018). https://www.interaction-design.org/literature/article/ prototyping-learn-eight-common-methods-and-best-practices

[6] Erica Dhawan and Tomas Chamorro-Premuzic. 2018. How to Collaborate Ef-fectively If Your Team Is Remote. HBR 27 (2 2018), 5. https://hbr.org/2018/02/ how-to-collaborate-e�ectively-if-your-team-is-remote

[7] Katherine M Everitt, Scott R Klemmer, Robert Lee, and James A Landay. 2003. Two Worlds Apart: Bridging the Gap Between Physical and Virtual Media for Distributed Design Collaboration. Technical Report. http://delivery. acm.org.proxy.uba.uva.nl:2048/10.1145/650000/642707/p553-everitt.pdf?ip= 146.50.98.29&id=642707&acc=ACTIVESERVICE&key=0C390721DC3021FF. 86041C471C98F6DA.4D4702B0C3E38B35.4D4702B0C3E38B35&__acm__= 1554922462_faf017e378ab9b9ca4328a49e26d77f9

[8] Mana Farshid, Jeannette Paschen, Theresa Eriksson, and Jan Kietzmann. 2018. Go boldly!: Explore augmented reality (AR), virtual reality (VR), and mixed reality (MR) for business. (9 2018), 657–663 pages. https://doi.org/10.1016/j.bushor.2018. 05.009

[9] Gerhard Fischer. 2004. Social Creativity: Turning Barriers into Opportunities for Collaborative Design. Proceedings of the 8th conference on Participatory design: Artful integration: interweaving media, materials and practices (2004), 152–161. http://delivery.acm.org/10.1145/1020000/1011889/p152-�scher.pdf?ip= 146.50.150.52&id=1011889&acc=ACTIVESERVICE&key=0C390721DC3021FF. 86041C471C98F6DA.4D4702B0C3E38B35.4D4702B0C3E38B35&__acm__= 1559569405_bfcb5a7796f2eb80a4124da63dadf44chttp://citeseerx.is

[10] David Forsyth, Rajinder S. Sodhi, Giuliano Maciocci, Brian P. Bailey, and Brett R. Jones. 2013. BeThere. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’13. ACM Press, New York, New York, USA, 179. https://doi.org/10.1145/2470654.2470679

[11] John S. Gero. 1996. Creativity, emergence and evolution in design. Knowledge-Based Systems 9, 7 (11 1996), 435–448. https://doi.org/10.1016/S0950-7051(96) 01054-4

[12] Susan Goldin-Meadow and Sian L. Beilock. 2010. Action’s in�uence on thought: The case of gesture. Perspectives on Psychological Science 5, 6 (11 2010), 664–674. https://doi.org/10.1177/1745691610388764

[13] Rae Jeneanne. 2015. Design Value Index. Design Management Review 26, 1 (2015), 4–8. https://www.dmi.org/page/2015DVIandOTWhttps://onlinelibrary. wiley.com/doi/abs/10.1111/drev.10307

[14] Robert E Kraut, Susan R Fussell, and Jane Siegel. 2003. Visual information as a conversational resource in collaborative physical tasks. Human-Computer Interaction 18, 1-2 (2003), 13–49. https://doi.org/10.1207/S15327051HCI1812{_}2 [15] Eunji Lee. 2016. Service Design Challenge Transitioning From Concept to Im-plementation. In ServDes 2016 - Fifth service design and innovation conference. –. http://www.ep.liu.se/ecp/125/019/ecp16125019.pdf

[16] Mary Lou Maher, John Gero, Lina Lee, Rongrong Yu, and Tim Clausner. 2016. Measuring the e�ect of tangible interaction on design cognition. In Lecture Notes in Computer Science (including subseries Lecture Notes in Arti�cial Intelligence and Lecture Notes in Bioinformatics), Vol. 9743. Springer, Cham, 348–360. https: //doi.org/10.1007/978-3-319-39955-3{_}33

[17] Rahul Malik, Melissa Dobosh, Marshall S. Poole, Kenton McHenry, and Peter Bajcsy. 2010. Evaluating the e�ectiveness of communication media on remote collaboration. In VECIMS 2010 - 2010 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, Proceedings. IEEE, 130–135. https://doi.org/10.1109/VECIMS.2010.5609355

[18] Chiu Mao-Lin. 2002. An organizational view of design communication in design collaboration. Design Studies 23, 2 (2002), 187–210. https://doi.org/10.1016/ S0142-694X(01)00019-9

[19] David Nguyen and John Canny. 2007. MultiView : Improving Trust in Group Video Conferencing Through Spatial Faithfulness. In CHI 2007 Proceedings. 1465– 1474. http://bid.berkeley.edu/�les/papers/multiview07.pdf

[20] Donald A. Norman and Roberto Verganti. 2014. Incremental and radical inno-vation: design research vs. technology and meaning change. Design issues 1, 1 (2014), 51–66. https://doi.org/10.1162/desi

[21] Sergio Orts-Escolano, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Christoph Rhemann, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, Shahram Izadi, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, and Sameh Khamis. 2017. Holoporta-tion. 741–754. https://doi.org/10.1145/2984511.2984517

[22] Tobias Pottho�, Dominik Siemon, Konstantin Wilms, Sascha Möser, Marco Hell-mann, Stefan Stieglitz, and Susanne Robra-Bissantz. 2018. Collaborative Service Blueprinting for Design Thinking: Evaluation of a Digital Prototype. In Proceed-ings of the 51st Hawaii International Conference on System Sciences. 4585–4593. http://redux.js.org/https://scholarspace.manoa.hawaii.edu/handle/10125/50467 [23] Giuseppe Riva. 2003. The Sociocognitive Psychology of Computer-Mediated

Communication: The Present and Future of Technology-Based Interactions. CyberPsychology & Behavior 5, 6 (2003), 581–598. https://doi.org/10.1089/ 109493102321018222

[24] Nancy B. Solomon, American Institute of Architects., and Martin and Margy Meyerson Accession Fund for the Built Environment Collections. 2008. Architecture : celebrating the past, designing the future. Visual Reference Publica-tions. xi, 407 p. pages. https://books.google.nl/books?id=ErvkPOjhulEC&dq= digital+tools+involved+in+design+process&hl=nl&source=gbs_navlinks_shttp: //hdl.library.upenn.edu/1017.12/366255

[25] Matthew Tait and Mark Billinghurst. 2015. The E�ect of View Independence in a Collaborative AR System. Computer Supported Cooperative Work: CSCW: An International Journal 24, 6 (12 2015), 563–589. https://doi.org/10.1007/ s10606-015-9231-8

[26] Mieke Van der Bijl Brouwer and Kees Dorst. 2016. How deep is deep ? A four-layer model of insights into human needs for design innovation HOW DEEP IS DEEP ? A FOUR-LAYER MODEL OF INSIGHTS INTO HUMAN. January 2014 (2016), 280–287.

[27] Erik Van der Pluijm. [n. d.]. Welcome to the War Room - Design a bet-ter business. ([n. d.]). https://designabetterbusiness.com/2016/12/02/ welcome-to-the-war-room/http://designabetterbusiness.com/2016/12/02/ welcome-to-the-war-room/

[28] Xiangyu Wang, Peter E.D. Love, Mi Jeong Kim, and Wei Wang. 2014. Mutual awareness in collaborative design: An Augmented Reality integrated telepresence system. Computers in Industry 65, 2 (2 2014), 314–324. https://doi.org/10.1016/j. compind.2013.11.012

[29] Andy Warr and Eamonn O’neill. 2005. Understanding Design as a Social Creative Process. http://delivery.acm.org.proxy.uba.uva.nl: 2048/10.1145/1060000/1056242/p118-warr.pdf?ip=146.50.98.29&id=1056242& acc=ACTIVESERVICE&key=0C390721DC3021FF.86041C471C98F6DA. 4D4702B0C3E38B35.4D4702B0C3E38B35&__acm__=1559817876_ a2e2e4a322f12eeb0ee84a7f9d531d84

[30] Martin Weinmann and Sven Wursthorn. 2019. Evaluation of the Microsoft HoloLens for the Mapping of Indoor Building Evaluation of the Microsoft HoloLens for the Mapping of Indoor Building Environments. February (2019). https://www.researchgate.net/publication/331311412

[31] Yandle. 2018. Design iteration brings powerful results. So, do it again designer! (2018). https://www.interaction-design.org/literature/article/ design-iteration-brings-powerful-results-so-do-it-again-designer

(13)

12 Appendix A. Interview guide

Introduction

Welcome and thanks for taking the time to do the interview with me. The current research aims to

study the remote collaborations in the context of design process. You are participating at the early

stage of my research process where I want to gain some insights into the design process as well as

how remote collaborations are achieved. The interview will take roughly an hour, and the response

will be used anonymously in the research study. In order for me to synthesize the findings after the

interview, is it okay if I do a voice recording of our conversation?

1. Tell me about your daily responsibilities and role at your job

Design process

2. Can you draw down a past design process? It could be for any projects, of any kind. If you

have trouble choosing one, just think about the last design project you were in. (Including a

few important interactions through the process, other parties involved, and physical

environment).

Collaboration

3. Based on your experience, how important is collaboration in the context of design?

4. Who do you collaborate with (might be a replicated answer as 4), what are their roles, and

in what context do you collaborate with them?

5. In the process of collaborating with others, what inspires you and what discourages you (i.e.

in case of environment, behaviors, and setups)?

6. Tell me about some challenges you encountered in the past experiences of collaborating with

other people in the design process.

7. Follow up question on 8: based on the challenges you mentioned, why...happened? How did

you feel at the moment?

Remote collaboration

8. How do you deal with the situation where not everyone in the group is physically present or

available at the same time?

9. If interviewee hasn’t mention technology, then ask What technology do you use when you

collaborate with people remotely?

10. How do you feel when using … for collaboration?

11. What inspires you or discourage you when you communicate with people remotely?

12. In an ideal world, tell me one thing you would like to achieve when collaborating with people

remotely through technology.

13. Have you exposed to any 3D visualization technology, such as VR, AR, or MR? How do you

think of using these tools to support remote collaboration?

(14)

13 Appendix B. Coding schemes

RQ2. The role of collaboration in the process of design

Research

question Participants Quotes

Collaboration within design development team

RQ2 N4 You just work as closely as possible because it's a lot of back and forth. It's not like car

manufacturing where first you build the engine, then you fix the doors-- it's not an assembly line that moves like this. You literally move in circles. It's like back and forth back and forth back and forth. It's like "z z z z z", until you reach something and then you build a prototype you get feedback lots of changes, and then it goes "z z z z z" . So it's the main thing in building technology products with good design and good user interaction is like you have to work closely together to combine business insights, technology, and design insights.

RQ2 D2 I think also between... Between colleagues internally if you're thinking from a consulting

perspective there's a lot of back end collaboration between colleagues of course in a design process that involves a third party as well

Define the problem

RQ2 D2 But then it's not-- Like we set the scope and now we're just gonna follow the process of

design and that's... I think that's not the right approach.

RQ2 N2 Yeah I think you setting a clear goal of the design process and that's a good beginning to

understand what are the limitations and what are--for example the really must-haves.

RQ2 N4 and then we keep digging deeper until we know exactly this is the problem, that is most

important to them. So our process really starts with deep understanding of the customer. And that's where I think design thinking is super important. So in the early phase of the project, it's called design thinking in terms of consumer. So really understanding the consumer better.

RQ2 D2 If you find especially in the beginning that the problem-- to find it isn't necessarily fully

correct. If you have the space to discuss that with the client that can be a difficult discussion but it-- Or it can even be one that you cannot have and you have to just stick to the problems scope defined by the client. So I think that's-- I think that's the main challenge.

RQ2 D2 I think in general that's the case but it's good to validate that and really scope it directly. I

think that's one of the key steps because otherwise the rest of the process is gone in the wrong direction

RQ2 D5 in the beginning to form a vision on the project and to get a good understanding of what the

actual problem of the client is, is really important. If you don't do that correctly, it's not gonna work, because you go to different direction. It's like if your starting point is here and you already have an idea hey we're going this way, you might get a little bit of this, but in the end you will end up there. Yeah but if it's actually should be this way then it's gonna be hard to from here go there you already shoot to go in the direction.

RQ2 D5 To get a good understanding of the problem. I think what is most important and what's also

the hardest for people is to truly listen to what people are saying and dig a bit deeper. So what they say is-- if you have a sureface when you just ask interview questions, people will like tell you this bit, but there is a lot, you know the iceberg. There is a lot under the surface, and sometimes you just need to dig a little bit deeper or give them some examples, because if they say yeah we want to be a very innovative hotel, OK I can say OK clear vision, but if I don't know what they actually mean with being innovative, what am I going to do? So we just also ask OK, does that mean that you want to have this, or does it mean that you have to get them like also form your own opinion on what they actually want to achieve. So I think that's that's the way that we really got into the right problem.

RQ2 N4 So they want to understand the problem as much as possible. Like deep three levels-- So we

call it three levels of why. So if someone says I want Uber for dry cleaning. You're like Why, Why do you want Uber for dry cleaning? And then there's given answer and then like why do you want that. And then you ask why do you want that? So if you ask why enough times, you get to the core of the problem.

RQ2 D2 One of the key phases is getting it right at the beginning so really setting the right scope, or

defining the right problem, or defining the right opportunity. I think that's a very key phase. So... And. Validating that because I think it's easy to define a problem