BUDI: Building Urban Designs Interactively - a spatial-based visualization and collaboration platform for urban design

(1)

Requirements for the Degree of

MASTER OF SCIENCE

in the Department of Computer Science

c

Xi (Nathan) Sun, 2020 University of Victoria

(2)

BUDI: Building Urban Designs Interactively

A Spatial-Based Visualization and Collaboration Platform for Urban Design

by

Xi (Nathan) Sun

B.Sc., Nanjing University of Aeronautics and Astronautic, 2003 M.Sc., Nanjing University of Aeronautics and Astronautic, 2008

Supervisory Committee

Dr. Yvonne Coady, Supervisor (Department of Computer Science)

Dr. Sowmya Somanath, Committee Member (Department of Computer Science)

(3)

ABSTRACT

BU DI (Building Urban Designs Interactively) is an integrated 3D visualization and remote collaboration platform for complex urban design tasks. Users with different backgrounds can remotely engage in the entire design cycle, improving the quality of the end result. In this paper, I

consider the trade-offs encountered when trying to make spatial-based collaboration seamless. Specifically, I detail the multi-dimensional data visualization and interaction the platform provides, and outline how users can interact with and analyze various aspects of urban design. In

BU DI, the display and interactive environment was designed to seamlessly expand beyond a traditional two-dimensional surface into a fully immersive three-dimensional space. Clients on various devices connect with servers for different functionalities tailored for different user groups. A demonstration with a local urban planning use-case shows the costs and benefits of BU DI as a spatial-based collaborative platform. A performance evaluation with remote collaboration shows

(4)

List of Figures

Figure 1.1 The BUDI System Running on a Touch Table and HoloLens . . . 5

Figure 2.1 Definition of Mixed Reality . . . 9

Figure 2.2 Microsoft HoloLens . . . 10

Figure 2.3 The Common Gestures Used In The HoloLens Application . . . 11

Figure 3.1 The Architecture of BUDI Client . . . 16

Figure 3.2 Cloud Based LAMP Web Server . . . 18

Figure 4.1 The Camera Frustum For Determining The Scope of The Map Display . . . 21

Figure 4.2 Satellite Map And Terrain Map . . . 24

Figure 4.3 Add Mesh Filter Component and Export Embedded Materials on FBX Models 25 Figure 4.4 ArchieveFileSystem Container . . . 26

Figure 4.5 A Server Based AssetBundle Stroage System . . . 27

Figure 4.6 Three Side Menus for Collaborative Design Task . . . 28

Figure 4.7 Four Ways Button For Moving A Case . . . 31

Figure 4.8 Using Two Fingers On The Object For Rotation . . . 32

Figure 4.9 Using The Closest Distance Detection For Object Rotation . . . 33

Figure 4.10 Buttons For Rotating A Case In HoloLens . . . 34

Figure 4.11 Investigation Windows For The Case Detail . . . 35

Figure 4.12 Drag A Rectangle Using Finger In Unity . . . 36

Figure 4.13 A Statistic Window For All Selected Cases . . . 37

Figure 4.14 Triangles Compose A Circle . . . 38

Figure 4.15 Using Five Vertices To Form A Sector . . . 38

Figure 4.16 FOV Simulation On The Map . . . 39

Figure 4.17 Altitude Angles And Position Calculation Sketch . . . 40

Figure 4.18 Settings For Timing And Location Adjustment On Design Area . . . 41

(9)

Figure 5.3 Performance During The Map Operation . . . 51

Figure 5.4 Satellite Map and Street Map in BUDI . . . 51

Figure 5.5 The Occlusion In The Terrain Map . . . 52

Figure 5.6 Moving A Map From The Floor to A Table in HoloLens . . . 53

Figure 5.7 Had A Closer Look on The Object in HoloLens . . . 53

Figure 5.8 Users Were Dragging Three Cases onto The Map Simultaneously . . . 54

Figure 5.9 Multi-Touch To Move 5 Cases With 5 Fingers At The Same Time . . . 56

Figure 5.10 It Is Impossible To Put More Than One Finger On A Small Scale Model . . . 57

Figure 5.11 Finger Conflicts During The Rotation . . . 57

Figure 5.12 Button For Case Manipulation . . . 58

Figure 5.13 Investigation Windows For The Case Detail . . . 60

Figure 5.14 Investigation Windows Occlusion . . . 61

Figure 5.15 A Statistic Window For All Selected Cases . . . 61

Figure 5.16 An Irregular Area For Selecting Cases . . . 61

Figure 5.17 FOV Simulation On The Map . . . 62

Figure 5.18 The open community with more building exposed . . . 62

Figure 5.19 A community with more privacy . . . 63

Figure 5.20 The FOV from a resident in the building . . . 63

Figure 5.21 The Illustration Of Sunlight And Shadow In The Dense Community During Summer Time . . . 64

Figure 5.22 The Illustration Of Sunlight And Shadow In The Dense Community During Winter Time . . . 65

Figure 5.23 The Back Row Building In The Spare Community Can Receive More Sunlight 65 Figure 5.24 Reduce Height Of The Front Row To Increase Sunshine For The Back Row . 66 Figure 5.25 The Task Changed From Victoria to Revelstoke with the Database Update . . 67

(10)

Figure 5.26 A Case Synchronization in Different Clients . . . 68 Figure 5.27 A Case Synchronization was presented on HoloLens . . . 68 Figure 5.28 The Cases Were Locked By An Operator and Frozen On Other Devices . . . 69 Figure 5.29 The Connection To Server Had Lost Due To The Serious Network Traffic . . 71 Figure 5.30 The Interface Of Teleconference Component For Remote Assistance . . . 72 Figure 5.31 The Performance When and After Running Teleconference Tool . . . 73

(11)

Last but not the least, I would like to thank my caring, loving, and supportive wife Nora, and my lovely kids Albert and Claire, for your encouragement when things got rough.

I sincerely thank my parents: Yongdong and Yiping for giving birth to me and supporting me spiritually throughout my life.

(12)

Chapter 1 Introduction

U rban planning is a comprehensive process to prepare urban construction, study the layout of a city for future development, and arrange various urban construction projects. It is a multi-faceted problem that requires the cooperation of different stakeholders to explore solutions [15]. Success-ful, collaborative, urban designs can improve the development of cities and sustainability over-all [17].

The primary process of urban planning includes 3 stages: information collection, design/im-plementation, feedback/finalize. During the information collection, designers need to collect re-quirements from stakeholders and overview the whole project, including the construction scale, environment, economy, etc., to come up with a rough idea of the planning. In the second stage, de-signers carry out concrete steps towards implementation according to the collected data and design principles. During future iterations, they continuously learn and brainstorm by obtaining advice from experts in various disciplines to polish their designs to meet the requirements. In the final assessment stage, designers receive feedback from industry experts and citizens, allowing them to optimize and finalize the ultimate solution.

In traditional urban planning, these three stages are usually isolated; each stage handled by various independent systems. In the first stage, a database is usually created to store the collected architectural, geographical and environmental data for a design proposal. The data is queried and retrieved in the design stage for decision making. Diverse applications are usually used for data analysis, model creation, and layout arrangement in the design stage. Many modelling applica-tions, such as AutoDesk’s AutoCAD , Trimble’s SketchUp,R are used to create architecturalR

models [33, 9, 38, 25]. Design software like Adobe’s PhotoShop often carries out the planarR

layout planing for two-dimensional communities[35]. Ecotect is a powerful design and dataR

(13)

cialists and stakeholders evaluate the design outcome afterward in traditional urban design proce-dures. The absence of experts in the design stage causes repeated assessments and modifications, which typically prolong the design. The quality of the outcome depends on the designers’ ability and experience. Therefore, we need to create a system for data storage, design, and multi-user interaction.

To address these problems, I implemented a new urban design platform BU DI to integrate data storage and analysis, architectural design, and multi-user interaction to blend three design stages into a unified system, where all participants can engage and cooperate on the same design task. The new system has a 3D design space to visualize data from multi-dimensional and heterogeneous information sources. The goals is to reduce the cognitive burden for people to understand spatial and geographic information in the design.

Participants may use the design system for various purposes. Designers can focus on the con-straints within the new urban setting, whereas experts and stakeholders can explore the overall solution demonstration. Different devices in BU DI meet these specific requirements. For ex-ample, designers can use an in-depth desktop application for their designs and implementation. Lightweight, portable devices can be used to demonstrate and illustrate the design solution to spe-cialists and stakeholders. A web-page display will be published to the public to show the final decision with less technical details. With virtual reality (VR) and augmented reality (AR) tech-nologies, the design solution can be presented in an immersive three-dimensional space, where users can experience and interact with the design. All of these can help various user groups to deepen their understanding of the multiple aspects of urban design [19, 24].

(14)

1.1 Motivation and Research Goal

Data visualization and interaction, as well as user collaboration, are the main features of BU DI. Specifically, I overview the motivation related to multi-dimensional data visualization, interaction and collaboration within the context of urban design in the following sections.

1.1.1 Multi-dimensional Data Visualization in Urban Design

Data visualization is effective for not only presenting essential information of data but also driving complex analysis [16, 18]. Data points in the multi-dimensional data sometimes have weak con-nections with each other. In urban planning, various independent data sets are involved, such as geographical data, environmental conditions and social factors. This data usually cannot be com-pletely expressed on a two-dimensional plane. Multi-dimensional data visualization uses compre-hensible visual elements to strengthen these connections, making data more understandable and facilitating data analysis and exploration. For example, an architectural model may contain a 3D model, pictures, and other illustrative data. The applications of three-dimensional urban architec-ture has become a trend in urban planning[36, 7, 28]. But how can 3D visualization be used in urban planning to show the complex-structured architecture data and spatial-based data is still a problem and bring the first research question.

Virtual Reality (VR) further simulates an immersive 3D virtual world to provide users with visual, auditory, tactile and sometimes even other sensory stimulation [21]. It provides interac-tion abilities and may help inspire design ideas in an immersive multi-dimensional space [27]. The emergence of virtual and augmented reality technologies brings more possibilities for multi-dimensional data visualization [27, 30]. The complexity of 3D data requires a hyper-multi-dimensional space for analysis and interaction. Compared to the traditional ”flat” display, an immersive en-vironment was shown to bring a better perception of geometry, intuitive data understanding, and retention of the perceived relationships in the complex data. Moreover, virtual reality can provide a more convenient and natural interactive way to support collaborative data exploration.

1.1.2 Interaction in Data Visualization

Interaction has been a consistent and well studied process in data visualization[23, 40, 13]. In-teractivity allows users to select, navigate and explore the data via an interactive interface so that users can understand specified attributes of data. An appropriate visualization strategy is critical to observe data that has complex structure and attributes, and a proper interaction is relevant to the

(15)

that was how to choose the appropriate devices for all user groups to adapt their roles in the urban planning and provide diverse interaction ways to facilitate user’s operations and conduct different design tasks.

1.1.3 Collaboration in Urban Design

Collaboration is a process and an ability to coordinate multiple individuals to achieve the same goal. In an urban planning context, all participants involved in the design have different back-grounds, so their expressions and views on the same plan might be inconsistent. Sometimes, these diverse backgrounds are complementary and conflicting in the design work. Thus, real-time collaboration and communication in the design process are particularly necessary to bridge their differences. The third research question is raised here, can we create a seamless collaboration in a urban planning for diverse purposes of different participants?

VR creates a fully immersive virtual world. Augmented reality (AR) seamlessly integrates the real world and the virtual scene. Augmented reality provides an open environment where users can interact with virtual data while observing and feeling the real world by normal senses, such as looking, touching, etc. AR has been applied in many complicated collaborative scenarios, such as medical treatment, and industrial design [6, 8].

In the AR context, collaboration can be categorized as face-to-face and remote. In face-to-face collaboration, users engage a task in the same place, with a shared perspective. The virtual scene is visualized in different personal devices through unified spatial coordinates reference. Users can interact with the virtual scene, sharing the results of their operations. Conversely, typically in remote collaboration, users participate from different locations, with different spatial references. The collaboration usually does not involve a multi-way scene sharing. In BU DI, I created a unified virtual environment to attempt to reconcile remote collaboration as a face-to-face spacial reference among all participants, local and remote.

(16)

Figure 1.1: The BUDI System Running on a Touch Table and HoloLens

1.2 Contributions

In this paper, to address these research questions, I present an approach to integrate complex-structured architectural data and geospatial information into a three-dimensional presentation with support for remote collaboration (Figure 1.1). I harnessed the power of the Unity game engine to create touch-table based interactive interfaces and an immersive mixed reality space on AR devices for the co-located interaction.

In BU DI, the prototype implementation of this approach, interaction and collaboration in ur-ban planning are facilitated as all participants are be connected to this system via cloud servers. Operational data is shared and synchronized among various terminals to guarantee data consis-tency. Virtual technology was introduced to enrich data interactions and exploration. With real-time communication over a network, specialists or stakeholders can seamlessly engage in urban design processes and actively bring their perspectives and background knowledge into the plan. BU DI is designed to integrate the otherwise isolated stages of urban planning to shorten the de-sign cycle, and improve collaboration in order to increase the quality of the final decisions.

1.3 Thesis Organization

In Chapter 2, I summarized the related background information and knowledge. Chapter 3 described the architecture of BU DI

In Chapter 4, I introduced implementation details of BU DI. I presented the essential algo-rithms, strategies of main components.

In Chapter 5, I designed a local urban design scenario to walk through BU DI to discuss the functionalities, rationality, performance and trade-offs of the implemented prototype.

(17)

In this chapter, I introduced the essential background knowledge, including data sources ElementDB, multi-dimensional visualization, and concepts of interaction and collaboration.

2.1 ElementsDB Database

The ElementsDB database that provides data and cases for urban design is hosted by the School of Architecture and Landscape Architecture in UBC (University of British Columbia, Canada). This database hosts information about real buildings and other landscapes (i.e. parks, roads) found throughout British Columbia, Canada. ElementsDB catalogues real examples of urban land uses and illustrates them by rigorous standards and graphic conventions. Various data are included, such as density, floor area ratio (FAR), building/site use and coverage. More illustrative data, such as photos, plans, and three-dimensional models, are also involved. Most information is collected from public sources and simplified for clarity and comparability.

ElementsDB is based on the concept that urban architecture is comprised of replicable elements or cases. Cases in ElementsDB, as natural multi-dimensional data, were directly imported into BU DI to evaluate the advantages of three-dimensional and virtual visualization. ElementsDB is still in development, regularly adding new cases and improving functionalities and interfaces.

2.2 ESRI and Map Service

The map solution in BU DI is mainly provided by ESRI (Environmental Systems Research Insti-tute). ArcGIS is a series of GIS software produced by ESRI, including abundant development tools

(18)

and services. Developers can utilize ArcGIS to produce specified geographic information through web services.

Developers usually do not want to abandon their existing platforms but to embed GIS services in the original application. For this purpose, ESRI provides extended map and image services in the ArcGIS server. Developers can use these services in customized ArcGIS clients in two ways: Server Object Interceptors (SOI) and Server Object Extensions (SOE) [14].

SOE allows users to create service operations to extend the base functionality of map or image services. Most SOEs provide maps and geospatial data through RESTful or SOAP web services. ESRI offers the ArcGIS map service via an official address1. This service gives a variety of map modes for choice, including the satellite and aerial map, street map, and terrain map. In BU DI , the base map was implemented in the SOE way.

ESRI also offers elevation service for terrain map. The elevation data can be downloaded from the web API2_{. Elevation service provides 17 levels of elevation data, and all map services offer 24}

levels of details.

In the ArcGIS map service, a tile is the smallest unit of a map. Tiles with different levels and resolutions can be downloaded through the RESTful API to form a customized map. As the map is usually displayed on a plane, it is necessary to convert the spherical coordinates of the earth into planar coordinates. Geographical scientists usually use Mercator projection to convert coordinates [29].

Sometimes we need to examine macro map information, such as the national boundaries of each country in the world map, and sometimes we need to investigate micro map information, such as the road condition during navigation. To this end, the map should be graded. At the lowest level (zoom = 0), a 256 × 256 pixel image is used to present the preview of the whole earth. At the next level (zoom = 1), the information increases more, presented by a 512 × 512 pixel image. Images in the higher level have four times as many pixels as those in the previous level. Each image is divided into numbers of 256 × 256 pixel sub-images, called tiles. There is only one tile at the zoom-0 level. There are four tiles in zoom-1 level and 16 tiles in zoom-2. By analogy, there are 4n_{tiles at the n level.}

Once the tiles are created, they will be labelled. ESRI labels a tile with an XY Z number [4]. Z represents the zoom level. The origin of (X, Y ) tiles is in the top left corner. X increases from left to right, meantime, Y grows from top to bottom. Once latitude and longitude are located, the tile number can be calculated from the zoom level by the following formula.

1_{https://server.arcgisonline.com/arcgis/rest/services}

2_{http://elevationthree-dimensional.arcgis.com/arcgis/rest/services/}

(19)

Virtual Reality(VR) is the technology of computer simulation to produce an immersive three-dimensional virtual world to provide users with visual, auditory, tactile and other sensory stimula-tion [21]. Users can experience this virtual environment and observe virtual objects in a timeless and spaceless way. As the user moves, the computer can instantly perform complex calculations, sending back accurate images of the three-dimensional world to create a sense of presence. Virtual reality integrates the latest development achievements of computer graphics, computer simulation, artificial intelligence, induction, display and network parallel processing. The core of virtual real-ity is modelling and simulation [34]. It integrates virtual information, provides interaction abilities and helps inspire design ideas in an immersive multi-dimensional space [27]. Immersion, interac-tion, imagination are the three essential elements of a virtual reality system.

Augmented reality (AR) technology is derived from virtual reality, which can seamlessly inte-grate the real world and the virtual scene[12]. The computer technology simulates virtual data to visible objects, which can be superimposed to the real world and perceived by the human senses to achieve the sensory experience beyond reality. In contrast to virtual reality, augmented reality tech-nology provides an open environment where users can interact with virtual data while observing and feeling the real world by normal senses, such as looking, touching, hearing, etc.

Based on virtual reality and augmented reality, the concept of mixed reality (MR) was proposed in 1994 [26]. Mixed reality includes augmented reality and augmented virtuality, which refers to the combination of real and virtual worlds to create a new visual environment (Figure 2.1). In this environment, realistic and virtual objects coexist and interact with each other. On of the most successful and popular mixed reality devices is Microsoft HoloLens. It can scan the surroundings and save the collected data as a virtual spatial model. Following built-in physical rules, virtual objects can interact with the spatial model to simulate the interaction with the real world[2]. Mixed reality extends the augmented reality, making interaction between virtual and real objects possible.

(20)

Figure 2.1: Definition of Mixed Reality

Mixed reality can also be regarded as a kind of generalized augmented reality.

In the era of PC, people interact with digital data through mouse, keyboard and touch device. These habitual interactive ways become impracticable in the three-dimensional virtual world, so new interaction methods are required.

In the virtual world, interactions became more diverse and closer to natural behaviours. Gaze is a fundamental way of interacting. The aiming point follows the user’s sight synchronously like a mouse cursor in the 3D space. When the aiming cursor points to a manipulable object, the click operation will be triggered after a short stare. Controller, gyroscope, spatial sensor and other advanced hardware were introduced in AR devices to carry out more complicated operations. For instance, the controller can combine with the gaze to perform some common actions, such as shaking, pushing, flipping, hitting, throwing, etc. The gyroscope and spatial sensor can track the body gestures to locate player’s position in the virtual world. Various hardware greatly enriches the operation, improves the interaction and enhances the immersion of virtual reality.

In addition to traditional hardware, lots of new interaction technologies were developed. Voice control is a natural interaction way. Microsoft, Google, and other technology giants have launched their voice recognition system. Moreover, motion capture has been introduced to replace the phys-ical controller. For example, Leap Motion corporation has developed a gesture tracking technology to interact with virtual objects without any handheld devices[39]. Finger gestures and voice control have been applied to conduct all operations in Microsoft HoloLens thoroughly. All these interac-tion ways can be applied together to accomplish more complex interacinterac-tions.

Human-Computer Interaction (HCI) is a big revolution in AR. It provides the ability to interact with the complex virtual elements. With the development of HCI technology and devices, AR applications have been initially applied in many complicated scenarios, such as education, medical treatment, industrial design [22, 6, 8].

(21)

2.4 Head Mounted Devices

HMDs (head-mounted devices) are the primary tool for VR/AR display and processing. The main-stream VR HMDs include HTC Vive, Oculus Rift, PlayStation VR and Samsung Gear VR. With these devices, users can experience the fully immersive virtual world. HTC Vive can connect to its LightHouse system to track the user’s movement, while the other HMDs require users to stay still to avoid hitting surroundings.

AR and MR devices project the entire reality in the scene, so there is no movement limitation for users. Microsoft’s portable holographic computer equipment (HoloLens) has achieved pro-duction and commercial sale (Figure 2.2). HoloLens presents the virtual holographic projection directly in the real world by using mixed reality technology to enhance the sense of reality. It provides a new way to interact with the real world.

HoloLens has seven cameras on the top of the glasses, among which four of them are used for spatial scanning. One camera at the middle top captures the user’s view. The two cameras at the center bottom locate the user’s sight and recognize gestures3_{. The computing module of HoloLens}

is composed of CPU, GPU, and HPU. HoloLens can run the program individually without external computing devices. Through the glasses, the user can see the real world, blended with virtual objects that follow physics, such as acceleration, gravity, collision and deformation. Users can interact with these virtual objects in the following three ways: Gaze, Gesture, Voice

The gaze is used for selection. When we look at an object, the ray casting from the middle-top camera will collide with this object to trigger a selection. Hand gesture detection is a main feature of HoloLens. Although hand gestures cannot be accurately positioned in the space, they have the advantage of allowing users to navigate through the HoloLens quickly without any other gear. HoloLens can recognize several hand gestures (Figure 2.3). The bloom and air tap are two

3

(22)

Figure 2.3: The Common Gestures Used In The HoloLens Application

primary ones. The bloom gesture stands for ”home” in HoloLens. This gesture represents a return to the start screen and has the same meaning as the Windows start button. Air tap is a gesture that requires the pointing finger to stand up and tap, similar to clicking with a mouse in the air. After the user gazes at a target, they can click through the air tap on it to trigger a command. The core gestures can also be combined into many new, separate interactions, such as two consecutive taps or tap-hold gestures. By combining the tap, hold, drag, we can express more complex and diverse meanings.

Voice control is also a significant feature of HoloLens. Voice is one of the most direct forms of human expression. In the virtual environment, some operations can be controlled by voice, which can compensate for the gesture accuracy and solve the problem of wrist pain caused by frequent gesture operations. Meanwhile, the hand gesture and voice control can complement each other to create various and convenient operations through the creative imagination by HoloLens users.

The powerful computing ability of HoloLens, as well as its portability, gives users great free-dom to participate in remote collaborative tasks anytime and anywhere. HoloLens has already been widely used in face-to-face collaborations and remote collaborations [10, 31, 5].

2.5 Teleconference and WebRTC

Teleconferencing has risen rapidly in recent years and is widely used in various industries. Tele-conferencing breaks the limitation of spatial and temporal restrictions. The user can participate in the meeting anytime wherever they are. Teleconferencing has significant advantages in commu-nication efficiency. There are many commercial applications for teleconference, such as Teams, Skype, Zoom, etc. Moreover, many open-source protocols, such as RTMP, WebRTC, and HLS, can be used for the private teleconferencing system.

WebRTC (Web Real-Time Communication) is a communication protocol and an API that sup-ports real-time voice or video conversations in a web browser. It was open sourced on June 1, 2011, and was included in the W3C recommended standards[3]. WebRTC provides the core tech-nology of video conferencing, including audio and video collection, codec, network transmission,

(23)

threaded operating system based on POSIX and Unix for free use and distribution. Apache HTTP Server is an open-source web server run by the Apache software foundation. It is one of the most popular web server software. The main program of Apache is called ”httpd”, which can run in most computer operating systems. MySQL is one of the best RDBMS (Relational Database Man-agement System) applications for data storage and manMan-agement. PHP (Hypertext Preprocessor) is a general-purpose, open-source scripting language. PHP absorbs the characteristics of C language, Java and Perl. It is conducive to learn and especially suitable for web development.

With Docker’s virtualization technology, the four parts can be packaged as a Docker image after installation and configuration. Developers instantiate a local copy or create an instance in the cloud docker container to build a web server. The four parts in LAMP can also be deployed separately in different servers. As traffic increases, server stress can be alleviated by cluster scaling and load balancing.

Cloud platforms are services based on hardware and software resources that provide com-puting, networking and storage capabilities. Compared to the traditional server, it has significant advantages, such as flexible configuration, scalability, reliability, high resource utilization and cost-saving.

2.7 Development Tools and Unity

Unity3D is a multi-platform integrated game engine developed by Unity Technologies, which al-lows players to create interactive content such as 3D video games, architectural visualization, 3D animation, etc. Unity can run on Windows, Linux, and Mac OS X. Software developed with Unity can be deployed to various platforms, such as Windows, UWP, macOS, PlayStation, Android and

(24)

so on. The Unity engine supports over 27 platforms5_.

Unity contains a hierarchical integrated development environment with the visual editor, prop-erties editor, and dynamic game previews. It can be used to develop applications quickly by import-ing three-dimensional models exported from mainstream modellimport-ing software like Maya, 3DMax, SketchUp. The SKP files made by SketchUp are the primary model format used in BU DI.

Unity uses C# as programming languages which has rich libraries. It is based on open principle and can be customized without modifying the source code. Unity is appropriate for small teams to quickly kick off a new project.

(25)

In this chapter, I introduce the architecture of BU DI. BU DI extends a two-dimensional urban design system allowing local interaction to take advantage of multi-dimensional data visualization and other interaction methods. Specifically, BU DI uses HoloLens technology and a touch table desktop for 3D visualization and collaboration. I established a Client/Server architecture, which allows various terminals to share data and simultaneously interact with the same urban design scene from multiple locations. Users can conduct numerous operations on holographic objects. These operations include creating, deleting, rotating, zooming, investigating, and so on.

I used Unity as a development tool for both the HoloLens and touch table client applications. On the server-side, I adopted a LAMP architecture to build a web server. Clients communicate with the server using the HTTP protocol to share and synchronize the operational data.

3.1 Architecture of Client Application

As a collaborative urban design application, multiple users can work on the design at the same time. They can move, rotate and investigate the objects independently and simultaneously. In the client, all architectural models shown are presented as 3D objects. The urban designer can refer to diverse information, such as building height, terrain, illumination, sunlight, the field of view, to make decisions. Various tools were implemented to visualize these pieces of information to facilitate design tasks.

In this architecture (Figure 3.1), the core component is the Unity engine, which provides the essential functionalities for 3D visualization and interaction, including creating a user interface, 3D object modelling, multi-touch, and spatial mapping.

(26)

map types according to the user’s requirements. Map styles can be switched within a satellite map, street map, and terrain map. The map is displayed with both two-dimensional and three-dimensional view in BU DI client.

A data management component was implemented for downloading the case information from ElementDB 1 _{or other remote urban data sources. The operational data is stored in the data}

structure defined in this component. Moreover, this component is responsible for communication with a remote server for data synchronization.

The new UI for BU DI using Unity’s graphic canvas component. UI elements can be easily imported or modified on this canvas. In addition, I applied the reliable multi-touch solution from Unity for multi-user cooperation in BU DI . Unity generates touch events, like enter, touch-drag, touch-release, for individual user, so this solution can drive and detect unlimited touches on the screen. Moreover, Unity supports various devices , like touch table, iPad, smartphones, Microsoft surface, etc, without making any compatible modification.

The interaction component provides the channel for interacting with the UI and objects in the scene. Various interacting methods are supported, including multi-touch, hand gesture control, and voice control. Each method is accommodated by different interaction tasks. The finger touch is mainly for manipulating the building models. The hand gesture is applied for the object control on HoloLens. The user can use voice for some simple operations like task switching.

In addition, the extension component was designed to be compatible with different terminals, so all potential devices such as the touch table, touchpads, VR/AR equipments can access the system. Based on this component, I customized two specialized client applications, for the touch table and HoloLens, respectively.

The touch table client is a 3D desktop application. All design elements, including the building models, map, control buttons, menus, etc., are shown as three-dimensional objects. Finger touch and hand gestures were introduced as the primary interaction approach for input and object ma-nipulation. To navigate the map on a limited-size screen, a slippy map was implemented as the base map layer. The map tiles are cached after being downloaded from the ESRI’s map server2to achieve a non-delay experience during the map navigation.

The HoloLens client in BU DI has some specific features, like spatial mapping, voice control and reality interaction, to provide a fully immersive experience for urban planning. HoloLens usually works as a demonstration tool rather than a design application in the design task. The main purpose of HoloLens is to visualize the synchronized design solution and provide a communication

1_{https://elementsdb.sala.ubc.ca/}

(27)

(28)

channel from a remote HoloLens user to local designers for assistance.

In HoloLens client, the MRTK (Mixed Reality Tool Kits) was imported to support the spatial mapping and reality interaction. Thanks to the spatial mapping ability, HoloLens can scan surfaces of the real object. The user can put the virtual element like a real object on these surfaces. The hand gesture and voice are introduced for more natural interactions. They are efficient for manipulating spatial objects, especially when a user is not able to reach the holographic objectives.

In addition, as a collaboration system, BU DI supports real-time communication among par-ticipants during the design process. This is especially useful for a remote collaboration. I imple-mented a video teleconference component in BU DI, which was based on W ebRT C. Participants can have a real-time conversation to express their thoughts about the design plan.

3.2 Architecture of Server

The server of BU DI is an essential part for data synchronization and operation coordination. It was implemented with a LAMP structure consisting of four parts: web server, file server, data server, and real-time communication server (Figure 3.2).

The web server runs the Apach httpd program to provide a web service for clients. It handles the requests from clients to store, query and update data. In the web server, PHP scripts parse the request via httpd service and access the database to complete the corresponding operation.Httpd server also provides the path mapping. It can map the URL in the request to a file server to fetch the corresponding resources like building models, pictures, etc.

The file server works as a file storage. It stores project resources and provides access paths for web server to acquire these resources. Once a web server assigns an URL towards the file path, the corresponding file can be downloaded by the client via the httpd service. The file server takes large space to store data and files. It requires large hard disk, but less system resources, such as CPU power and memory. Therefore, it can be deployed in a cascaded hard drive with a small size system.

The data server was set up to provide running data storage and management. Urban design is a process with many negotiations and adjustments, the design is constantly evolving. Users save the task snapshot anytime to avoid data loss. Data consistency and security are primary considerations in BU DI. The MySQL relational database is installed to provide the data service for web server, where PHP scripts use the PHP-MySQL driver to access the database. The data in the database is generally written on the hard disk. However, the frequent hard-disk writing consumes system resources, like CPU, memory and hard disk I/O. To mitigate the competition with the web server,

(29)

Figure 3.2: Cloud Based LAMP Web Server

the data server can be deployed in anther physical machine. The web server communicates with the data server in a local network to reduce the network latency, so does the file server.

The real-time communication (signalling) server provides peer to peer connections via We-bRTC protocol. The server program is an executable command-line which accepts socket connec-tions on port 8888.It creates a channel for a pair of peers when generating a call. This is an open source program by Microsoft3.

Each server undertakes different roles in the entire back end system. They were separately de-ployed in different virtual machines. The web server, file server, and data server can be separately deployed on Ubuntu VMs. The web server can be accessed via port 8080. The signalling server can be deployed on a standard Windows virtual machine to provide the teleconferencing service in BU DI.

3

https://github.com/microsoft/WebRTC-universal-samples/tree/master/Samples/ PeerCC-Sample/Server

(30)

Chapter 4 Implementation

In this chapter, I applied multiple algorithms and strategies for each component in the implementa-tion of BU DI to provide various tools to facilitate designers’ urban planning and analysis. These mainly focus on 3D visualization and spatial interaction. Moreover, I implemented the network interfaces between client and server for data synchronization to achieve a real-time collaboration.

4.1 Design Strategies in Clients

Essential components in BU DI are the foundation of the collaborative design system and provide the basic operation and logical process for urban design tasks. These components are map gener-ation and control module, data management module, case genergener-ation and interaction module , as well as miscellaneous tools including field of view tool and sunlight simulation tool.

4.1.1 Map Generation and Control

I created a slippy map as a base map layer for urban planing task. User can control the map, like zooming and panning, by their fingers. The map was formed by a bunch of tiles downloaded from ESRI’s map service. Once the longitude, latitude and map level of the plan area were deter-mined, the corresponding map tile and elevation data can be retrieved through the UnityWebRe-quest method.

Tiles are loaded as textures in Unity and set to a mesh component to visualize a map. A center tile index is presented with a X-Y value. It can be calculated from the specified geolocation at a given zoom level with the following Coordinates − tileIndex formula1. The geolocation is

(31)

to ESRI’s host to download tiles. A list was defined to store all calculated indexes and indicate the downloaded tiles. When scrolling back to the previous zoom level or location, the application will check if the tile indexes are in the list before downloading the new tiles. This caching mechanism reduces duplication and network load and makes the map operation more smooth.

If we do not set any constraints when deriving tile neighbours, it may enter an infinite loop and download large amount of tiles of the entire earth. This consumes lots of time and storage and is not practical. We only need to display the map imagery in the screen area. I used a method

GeometryU tility.CalculateF rustumP lanes(M ainCamera)

from Unity to calculate planes that form the virtual camera’s view frustum (figure 4.1). In Unity, this camera capture the scene and display all objects in its frustum on the physical screen. When generating a tile object in Unity, I attached a collider component on this object. With the method:

GeometryU tility.T estP lanesAABB(planes, Collider.bounds),

the colliders’ boundary can be detected to determine if the tiles are included in the virtual camera’s frustum. All tiles out of this frustum are erased even if they have already displayed in the previous round.

Map Panning

The map panning can be implemented with finger dragging as the input. The finger touch can cast a ray in the direction of the virtual camera. When the ray hits the map plane, a collided position

2_{http://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/}

(32)

(33)

The EPSG900913 coordinates of the old map center can be calculated from its WGS84 coordi-nates (longitude and latitude). I used the .NET Spatial Reference and Projection Engine (Proj.NET) to perform this point-to-point coordinate conversion4_{. By adding the displacement, new central}

coordinates can be derived to form an updated map area, which can be considered as the map panning.

Map Zooming

The map can respond to two fingers pinching for zooming and two fingers turning for rotating. When we try to zoom or rotate the map, we do not actually control the map object itself. A virtual camera in Unity captures the view of the scene, and we adjust the virtual camera’s position and direction which makes the view change towards the map by the following formula.

cam.transf orm.localP osition− =

V ector3.up × (M anipulationGesture.DeltaScale − 1f ) × ZoomSpeed (4.3) DetalScale is a property of the M anipulationGesture class, which is implemented in a multi-touch library in Unity5_{. This property indicates a value of pinching direction and distance.}

Fingers moving apart is represented with a greater-than-1 value and fingers moving closer is repre-sented with a less-than-1 value. When we move two fingers apart, the camera’s height will reduce to make a closer look at the map. ZoomSpeed is a constant to adjust the zooming speed. It is set as 0.25f as a proper speed.

All downloaded tiles have different resolutions based on the map level. If we are too close to the map, the map will appear fuzzy. Therefore, we should dynamically increase the map level to

3_{https://wiki.openstreetmap.org/wiki/Zoom_levels} 4_{https://www.nuget.org/packages/ProjNet}

(34)

get a higher resolution during the map zooming.

Unity refreshes the scene at a frame rate. The map level changes with the zooming speed every frame by the following formula. In the formula, a positive value of ZoomSpeed represents zoom-in operation. T ime.deltaT ime is the zoom-interval between two frames, and a constant 4.0f controls the increment. The level value keeps accumulating every frame with the finger pinching and is rounded to an integer. Eventually, a new level value will trigger a new round of tile downloading to form a map with different resolution.

CurrentLevel+ = 4.0f × ZoomSpeed × T ime.deltaT ime

M apLevel = int(CurrentLevel) (4.4)

Adjusting the virtual camera’s position and orientation can bring various perspectives in Unity’s 3D spaces. With a proper adjustment, a three-dimensional perspective view on the map can be introduced to investigate the spatial relationship of buildings.

Map Control In HoloLens

In the HoloLens application, the virtual camera is mapped to a physical camera on the HoloLens and dynamically moves with user’s motion. It is hard to calculate the boundary of the map to limit the tile quantities. So the map in HoloLens is formed with a fixed number of tiles.

I proposed to create a terrain map to visualize the impact of terrain on urban design. A terrain map is usually implemented through a terrain component in Unity. With the above Coordinates − tileIndex formula, the central tile’s index can be calculated by its coordinates, then indexes of a 4 × 4 tiles square around the central tile can be derived. All 16 tiles’ imagery are downloaded as two-dimensional textures and set to the property SplatP rototype in the terrain component to visualize the map image. Besides the tile imagery, the elevation information of each tile will also be retrieved from ESRI’s elevation service 6_{. This information is encoded as a pixel type file.}

An ESRI official library LERC.dll can decode this file to an elevation structure. It includes a 256 × 256 height array, which indicates the height of each pixel of a map tile. For 16 tiles, the elevation structure is composed of all height arrays of each tile together to form a 1024 × 1024 array.

However, the map’s resolution in the terrain component are usually not equal to the size of the elevation structure. In BU DI’s HoloLens application, the height map resolution is set as 256×256. The 1024 × 1024 array need to be scaled to a new 256 × 256 array by calculating the average value

6_{http://elevationthree-dimensional.arcgis.com/arcgis/rest/services/}

(35)

Figure 4.2: Satellite Map And Terrain Map

of each 4 × 4 points in elevation structure, so that the terrain component can apply the elevation in-formation on each pixel of the map. The method terrainData.SetHeights(0, 0, HeightArray()) is invoked to accomplish the elevation setting. Figure 4.2 shows a comparison of a plate satellite map and three-dimensional terrain Map. Building a terrain map is more complicated than building a mesh map. In the next chapter, I will evaluate both mechanisms in performance and applicability.

4.1.2 Data Management

The data management is implemented via a case maintenance tool. This tool is responsible for original case downloading and model pre-processing. A case represents an architecture instance in ElementDB. This representation includes an architecture model, pictures and illustrative data. It can be used repeatedly in different urban design tasks.

The tool connects to the ElementDB database7to download the case models and their illus-trative information. When the tool runs, it retrieves the case summary that indicates all available cases in ElementDB. Then, the tool downloads all corresponding resources, including case

(36)

Figure 4.3: Add Mesh Filter Component and Export Embedded Materials on FBX Models

els and preview images, from ElementDB storage. These resources are categorized and stored locally into a different directory.

The maintenance tool also pre-processes case models. The original downloaded models do not contain the collider and ridgebody components, which are used for object manipulation. There-fore, after they are downloaded, all three-dimensional models are added with mesh collider and ridgebody components which can receive touch events during the object manipulation. Moreover, original models use embedded textures and materials to visualize building appearance. Unity can not read these embedded materials. In Figure 4.3, the original model lacks details of the trees. So the internal materials and textures have to be exported and imported with the original model into Unity. With the separately imported materials and textures, Unity can display trees in the view.

After the model downloading, the illustrative information of cases, such as permeability, occu-pancy, land usage, etc., are retrieved via the ElementDB’s web API8. An XMLDocument library is used to parse the returned message and save the parsed fields into a list ResourceList. This list is eventually serialized to a JSON file and stored locally for case detail display.

However, the size of the three-dimensional model is often large. This occupies much local storage. With more cases added into the ElementDB, the total volume of models grows bigger

8

(37)

case models as AssetBundle files and uploads them to a file server to free the local storage. An AssetBundle is an archive file that contains platform-specific non-code assets, such as mod-els, textures, prefabs, audio clips, and scripts, which Unity can directly load at run time9_{. An}

As-setBundle contains multiple data as shown in Figure 4.4. AsAs-setBundles can express dependencies between each other. For example, a material in one AssetBundle can refer to a texture in another AssetBundle. To pack an AssetBundle file, an underlying object of a case model and all connected assets are included.

Unity supports two compression methods for AssetBundle packages: LZMA and LZ4. LZMA has the maximal compression rate, but with the lower loading speed. LZ4 compression is a ”chunk-based” algorithm, it achieves a faster decompression with a lower compression rate. After com-pression, Assetbundle files are uploaded to a remote file server to save the local storage. The client only downloads them when they are required in urban design. After loading, AssetBundle files will be cached into the memory until a corresponding object is instantiated.

I implemented an AssetBundle maker function in the case maintenance tool to pack and upload AssetBundle files to the remote server (Figure 4.5). As cases increases, more AssetBundles will be added into the server with this tool.

On the other hand, even though the internet is very speedy and stable today, there is latency when loading the AssetBundle due to network traffic and file size. In Chapter 5, I will evaluate the influence of the internet on the AssetBundle loading and explain why I need to apply a remote loading solution for collaboration.

(38)

(39)

4.1.3 Case Operations

After the base map is generated, we can visualize and organize cases on the map for urban design. The touch table application focuses on the layout design, while HoloLens emphasizes the visu-alization. In a urban planning scenario, designers gather around the touch table to present their proposal by placing the architectural models in the planning area. Properties of the building, like the position, orientation and size, are frequently adjusted with users’ operations. Multiple users use the touch table simultaneously and expect to manipulate different cases at the same time without any conflicts. Therefore, an independent case-control mechanism is used to avoid operation con-flicts. In this section, I introduce different strategies to process the case generation and interaction on each device.

Case Placement

The touch table application aims at providing the experience for an urban design squad to practice their planning ideas together. Multi-user collaboration is a basic requirement in a group. I designed three side menus on the touch table (Figure 4.6) to support the collocated operations. Users can drag several cases simultaneously onto the map from these case menus.

The user interface (UI) and the map are implemented on two different layers: UI layer and map layer. In Unity, the layer is used to categorize some similar objects. These two layers are placed on two parallel planes.

The UI layer is rendered and overlaid in the screen plane. All menus are displayed in this layer. In the case selection menu, all available cases are shown with their preview images (Figure 4.6). Users can select the cases they need from this list intuitively and drag their images out of the menus. These images move in the screen plane following the finger movement.

(40)

Every point on the screen area responds to a map location. A virtual ray is cast along the direction from the virtual camera to the finger touch point. When the ray hits the map plane, a collision point will be obtained as a map location. As the finger releases, a building model will be downloaded from the file server as an AssetBundle file and instantiated to a 3D object on the map. The building’s location on the map is converted to a W GS84 coordinate (longitude, latitude) and updated to BU DI server to share with other devices.

The HoloLens application is designed for visualizing the design layout. Cases shown on the HoloLens usually have been arranged in a touch table application. The case selection menus are removed in the HoloLens application. The HoloLens mainly manifests the states of these cases which are recorded in a remote database. The state data include Case IDs, geolocations(longitude, latitude), orientations of all existing cases. A synchronization script is implemented to continually check the state from the database. When the HoloLens application recognizes newly added cases, it will download the corresponding AssetBundle files and load them on the local map. If the data of any case changed by other terminals, the script can update the local status of relevant cases. This script is also used in the touch table application for case synchronization. The synchronizing process is discussed in detail in Section 4.3.

When the application retrieves the latest data of a case, the new geolocation of the case can be converted to the local map position with a coordinates proportional relation. First we get the coordinates of the upper-left and lower-right tiles of the map. In the section 4.1.1, I used a central tile to derive the neighbour tiles to form a 16 tiles HoloLens map. Once the indexes of the upper-left tile and lower-right tile are derived, their corresponding longitudes and latitudes can be calculated by the following formula, which is the reverse way of the Coordinates − tileIndex formula.

n = 2zoom

lat rad = lat deg/180 ∗ π

lon deg = x tile/n × 360.0 − 180.0

lat rad = arctan(sinh(×(1 − 2 × y tile/n))) lat deg = lat rad × 180.0/

(4.5)

In the HoloLens, the axis Y is orthogonal to the map plane, the X,Z axis represents the map plane. The case position on the map can be presented by a X-Z value. Since we already have the geolocation of the case and the coordinates of upper-left and down-right tiles. The X-Z value can be calculated by the following formulas. The constant SIZE represents the actual side length of the map in the HoloLens’ coordinates system.

(41)

and moves, the model’s position will be adjusted to the new map location where the finger points. As I discussed in UI layer, a ray can be cast to the map from the touch point. The building model can be detected by the ray. The ray will be terminated once it collides with an object. In Unity, everything in the scene, including the map and case models, can be considered as an object. Since the ray always meets the case model before the map, locations of the map area covered by the model object will never be obtained. So when the finger move above the case model, the new map location can not be detected. Once the finger leaves the model, the model will lose the connection with the finger. As a result, the case model can not be moved.

To solve this problem, I put all cases on a new layer and create another ray, which casts from the touch point following the same path of the first ray. The first ray detects the case model on the new layer, while the second ray ignores the new case layer and detect the map locations on the map layer. With this method, we can get the map location under the case model. When we point the finger to a case and move the finger slowly, the case’s position will be adjusted to the new map location with the finger’s movement.

The case moving is implemented by a C# script attached on every case object. Every script can detect the corresponding finger touch and run independently, so designers can operate on multiple objects simultaneously without any conflicts.

In the HoloLens application, moving a case with the finger movement is not a easy operation. The finger in the air is not stable enough to move in a small range. It is a challenge for fine-tuning a case model. Alternately, I designed a button menu to manipulate the object (Figure 4.7). This menu controls the case’s position by pressing different direction buttons. The adjusting unit of each button press is fine, so the designer can make accurate adjustments to the models.

(42)

(43)

Figure 4.8: Using Two Fingers On The Object For Rotation

Case Rotation

Building’s orientation is an essential consideration in urban planning. Case rotation usually re-quires a two-finger turning gesture. I used a multi-touch library10 _{to handle the complex gesture}

interaction on the touch screen. This library provides several individual gesture components that can attach to the object to control it. These components can detect the number of fingers on the corresponding object (Figure 4.8). The T ransf ormGestures component recognizes the rota-tion gesture with two finger turning. The T ransf ormer component receives rotarota-tion data from T ransf ormGestures and applies them to an object. By using this library, the rotation can be driven by two fingers turning on a building surface.

This approach needs two fingers on the same surface. It is practical for a large screen device like a touch table, because models usually appear big enough. But, when the application runs on the small size screen such as a laptop or tablet, the building model is usually too small to accommodate two fingers. It is difficult to apply this rotation.

As an alternative approach, I implemented an algorithm to detect two adjacent fingers for rota-tion. The finger touching on the case confirmed the selection, and the adjacent finger is detected by measuring the distance towards the selecting finger. In this way, the rotation can responds to these two fingers’ turning for a small object (Figure 4.9). There might be multiple fingers on the screen in such a collaborative application. How to determine the adjacent finger becomes a problem.

Normally, we use thumb and the index finger to rotate an object. So as not to affect other users’ operation, we usually put these two fingers as close as possible. Based on this assumption, I proposed a distance sorting algorithm to determine the closest finger towards the selecting one as

10

(44)

(45)

Figure 4.10: Buttons For Rotating A Case In HoloLens

the adjacent finger from all candidates.

The algorithm was divided into two steps. In the first step, a ray-casting method was used to detect the selecting finger. In the second step, the shortest distance from the other touch finger to the selecting finger were sorted out (AppendixA.1).

Distances can be calculated by the Unity function V ector2.Distance(end, start). These dis-tances were stored in an array, and the index of array is the finger ID. I used a bubble sorting algorithm to find out the closest finger as the adjacent one in a valid range. The valid range re-stricts the distances of candidate fingers to avoid the mismatch.

However, in HoloLens applications, rotation by finger gesture is not suitable. Alternately, I also used a button menu to adjust the case’s orientation (Figure 4.10) like the moving operation. I set the rotation angle as small as possible to ensure accuracy.

Case Detail Display

Urban design needs to consider many situations. It not only refers to the architecture’s geograph-ical information, but also many practgeograph-ical references, such as building types, usage, floor space, occupancy rate, etc. So, I implemented a displaying tool to explore these details during the design. All relevant details of the cases have been downloaded and saved into a JSON file by the case maintenance tool. When double-tapping on a case model, the corresponding information will be read from this file and displayed in a popup window over the model. Sometimes, the case information might be updated frequently in the server. To keep up with the data, I also provides an interface to query the latest data from ElementDB in this tool.

A display window was implemented as a scrolling table when there are many contents to show. Users can use the finger to scroll the text up and down to check the complete information (Figure

(46)

Figure 4.11: Investigation Windows For The Case Detail

4.11).

Case Statistics

Generally, designers design a community with a cluster of buildings and landscapes with different properties. According to these various sources of information, such as land occupation, population, greening rate, the designers can modify the architectural model, re-plan the building layout, and add greenery to achieve a more rational design.

The case displaying tool only works for a single case. A limited-size screen cannot accommo-date too many windows at the same time. Moreover, it is not practical for designers to summarize the case data one by one. Therefore, I implemented a multi-selection tool for data collection. Cases can be selected through this tool, and their information is grouped and displayed on the screen.

In this tool, a rectangle is defined by dragging the finger to create a selection (Figure 4.12). The start and the end points of the finger movement define the diagonal of the rectangle. These points are represented by two-dimensional vectors on the screen plane. I used the GL component in Unity to draw the rectangle.

(47)

(48)

Figure 4.13: A Statistic Window For All Selected Cases

these objects on the screen plane, their three-dimensional coordinates have to be transformed to two-dimensional vectors. The Unity method Camera.W orldT oScreenP oint(V ector3) was used to perform this transformation. In this way, the screen positions of all visualized cases on the map can be obtained. When these positions fall within the drawing rectangle, it means that these cases are selected. All selected models are highlighted with red color to distinguish from the other models. The information for each selected case can be read from the local JSON file or retrieved from ElementDB. All data will be summarized into a list and shown in a window on the right side (Figure 4.13).

4.1.4 Field of View and Sunlight System

In the modern metropolis, tall buildings stand side by side. Scattered buildings block each other out. People in some buildings do not even see the sun all year round. Therefore, architects and designers need to optimize the layout of buildings to solve the mutual shielding problems. To support the design optimization, I designed a field of view (FOV) tool to visualize the real sight view and a daylight simulation tool to simulate sunlight illumination on the building models.

FOV

Field of view (FOV) is an observable open area that a person can see through the eyes or an optical device. It is a coverage of an area in a certain radius. In BU DI ’s touch table application, the map is usually shown with a top-down view. The simulated FOV tool is beneficial to provide a different perspective for users to see the building layout and perceive the visible and invisible areas from the map plane. In a virtual environment, a FOV is essential for getting an immersive, life-like

(49)

Figure 4.15: Using Five Vertices To Form A Sector

experience.

I drew the The FOV sector by the Unity Mesh component. The basic unit of Mesh is a triangle, and many triangles compose a shape. We know a circle is composed of a group of triangles (Figure 4.14). When the number of triangular partitions is large enough, the short side of each partition approximates an arc. These short sides add up to form a circle. Similarly, to draw a sector is to draw a bunch of triangles. Three variables are needed: the angle, the radius and the number of partitions. The angle of each partition can be divided by the sector angle and the partition number. Three vertices make a triangle, and the sequence of triangles determines the shape. For exam-ple, as shown in figure 4.15, five vertices {{0,0,0},{-2,2,0},{-1,3,0},{1,3,0},{2,2,0}} were used to draw an approximate sector formed by three triangles. An array recorded the sequence of the vertices’ indexes , like [0,1,2,0,2,3,0,3,4]. Every group of three indexes in the array presents a triangle, such as [0,1,2], [0,2,3], etc. The property mesh.vertices from Class M esh in Unity was used to store all vertices’ positions in M esh component, and the property mesh.triangles was used to record the order of vertices and triangles (Appendix A.4).

(50)

Figure 4.16: FOV Simulation On The Map

To draw a FOV, I selected a view point on the map as a sector center. Then I defined the radius and a view angel with the specific direction. After that, I cast as many virtual rays as possible from the sector center in this range. As these rays reach the edge of the sector or some objects, the collision points were obtained as a vertices array. Meanwhile, the vertices and triangles’ se-quence were recorded. These vertex positions and sese-quence arrays were set to mesh.vertices and mesh.triangles to complete the sector drawing (Figure 4.16).

Daylight and Shade

A significant impact of the building layout on the residents is daylight. In some high latitude areas, the solar altitude angle is very low. If the urban planning does not carefully consider the impact of building layout on daylight time, the lower floors in high-rise buildings may not receive enough sunlight.

There are many algorithms to calculate the local solar altitude and angle[20, 11, 32], but they only provide references of daylight effect on individual buildings. Moreover, the sunlight exists in a three-dimensional space. It is impossible to demonstrate the sunlight in two-dimensional urban planning.

I took advantage of Unity’s lighting system to create a directional light source. This light source can simulate the realistic daylight and project the shade of a three-dimensional object on the surrounding objects. The shade changes according to the position and direction of the direc-tional light. Shadow has been approved to be the best reference to learn the impact of daylight on buildings [37]. The altitude angle and direction of the sunlight can be calculated based on the geolocation and time of the design area (Figure 4.17).

(51)

(52)

Figure 4.18: Settings For Timing And Location Adjustment On Design Area

Figure 4.19: Visualizing The Shadow Of Buildings

I implemented a tool to simulate the impact of building layout on sunlight receiving. In this tool, I applied the Azimuth and Altitude algorithm 11 to calculate the altitude angle and position of the sun. The designer can flexibly explore the sunlight condition at a specific area by adjusting geolocation and time. I also designed a time controller to adjust the daylight time (Figure 4.18). In this controller, the ”Time Speed” controls the simulation speed. With a high speed setting, the simulation system can demonstrate the daylight changing in a very short time.

When the tool is running, daylight is simulated and irradiates at the buildings to generate the shades. Figure 4.19 shows a snapshot of a building’s shades on its surrounding buildings (Figure 4.19). When designers change the building’s positions, the simulation continues with the new lay-out and generate new shades. Designers can intuitively optimize the architectural laylay-out according to these real-time shades.

4.2 Specific Features in HoloLens Client

HoloLens has many exclusive features in portability and immersive interaction, such as spatial mapping and voice control. In BU DI, it plays an important role to demonstrate the design solution and provide a channel for remote engagement for participants.

BUDI: Building Urban Designs Interactively - a spatial-based visualization and collaboration platform for urban design

Contents

List of Figures

Chapter 1

Introduction

1.1

Motivation and Research Goal

1.1.1

Multi-dimensional Data Visualization in Urban Design

1.1.2

Interaction in Data Visualization

1.1.3

Collaboration in Urban Design

1.2

Contributions

1.3

Thesis Organization

2.1

ElementsDB Database

2.2

ESRI and Map Service

2.4

Head Mounted Devices

2.5

Teleconference and WebRTC

2.7

Development Tools and Unity

3.1

Architecture of Client Application

3.2

Architecture of Server

Chapter 4

Implementation

4.1

Design Strategies in Clients

4.1.1

Map Generation and Control

4.1.2

Data Management

4.1.3

Case Operations

4.1.4

Field of View and Sunlight System

4.2

Specific Features in HoloLens Client