Measuring accessibility : an empirical comparison of accessibility measures in house pricing models in The Netherlands

(1)

Measuring Accessibility

An Empirical Comparison of Accessibility Measures in House

Pricing Models in The Netherlands

Matthijs Bootsman (10631372)

August 2017

MSc in Econometrics: Big Data Track

Faculty of Economics and Bussiness

University of Amsterdam

Supervisor UvA: dr. N.P.A. van Giersbergen Second Reader UvA: dr. J.C.M. van Ophem Supervisor Lynxx: MSc Kevin Mann

(2)

i

Preface

This master thesis was written at the Faculty of Economics and Business at the University of Amsterdam and at Lynxx headquarters. Lynxx is an Amersfoort-based data company, which provided me with a lot of data to work with. The research topic came up after discussing the work Lynxx has done on an accessibility project in the North of The Netherlands, which was commissioned by Connexxion.

Despite the close cooperation between all parties, this document is solely written by Matthijs Bootsman, who declares to take full responsibility for the contents of this document. I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it. The Faculty of Eco-nomics and Business is responsible solely for the supervision of completion of the work, not for the contents.

Amsterdam, 12-8-2017

(3)

ii

Acknowledgement

I would like to thank all the people at Lynxx for their kindness and helpfulness from my first day on. I learned a lot over the course of a few months and that is fully due to the amazing people that I worked with.

In particular, I want to thank the following persons. Sanneke Mulderink and Paul Rooijmans, for giving me the opportunity to write my thesis in the warm and challenging atmosphere of the Lynxx company. Simon Langbroek and Peter Nijhuis, for letting me in on their original project and the patience they had when including me. A great and special thank you goes out to my su-pervisor at Lynxx, Kevin Mann, for his many contributions and close involvement in this project. This thesis could not have been written without you.

Also, I am grateful for all that the University of Amsterdam has done for me over the years. Studying at the University of Amsterdam broadened my view and shaped me to what I am today. Special thanks goes out to my thesis supervisor at the university, Noud van Giersbergen, for the many contributions he had to this thesis during our pleasant conversations.

(4)

Chapter 1 Introduction

The relationship between accessibility and land use has been studied intensively in the past, but has drawn some renewed attention following the skyrocketing of housing prices in larger cities across the globe. Main statements regarding the influence of accessibility on land use in leading handbooks such asAlonso(1964) are still relevant today, although the interpretation of the term accessibility has been subject to changes over time. With the growing influence of public trans-portion on accessibility, a measure capturing just the absolute distance between the origin and destination does not suffice anymore. More recent studies concerning accessibility have there-fore made use of more specific models, see for example the GIS-based estimation byBenenson et al.(2011) in the metropolitan area of Tel Aviv. The researchers find, among other things, that using actual travel times as a measure of accessibility yields more useful results than crudely using the absolute distance in kilometers. Especially in metropolitan areas, where the use of public transportion can help to avoid rush hour peaks on the road, the usage of net travel times instead of absolute distances seems justified, but this approach has not been widely adopted yet.

With the introduction of trip planning software such as the directions feature in Google Maps, gathering data containing actual travel times has become much easier. As most of the trip plan-ners use both GPS and GTFS data, it is possible to compare different ways of travel, such as pub-lic transportation and transport by car. Despite the availability of the data, modelling accessi-bility using net travel times has not been widely used in recent studies. For example,Armstrong

(7)

CHAPTER 1. INTRODUCTION 2

and Rodriguez (2006) use proximity to train stations as a measure of accessibility, whereas Ia-cono and Levinson(2011) do use net travel time, but limit their research to highways in a county in Minnesota. A comprehensive study that assesses actual travel times and accessibility on a state or nation wide scale is missing, as combining data from different counties or metropoli-tan areas gives rise to uniformity issues. However, given the small number of transportation providers and the high coverage rate they together have, The Netherlands is suited for creating a uniform accessibility measure for areas throughout the country.

Such an accessibility measure is often based on the number of available jobs that can be reached within a fixed distance or time. In previous studies, transportation is specified to have a homo-geneous unit cost and universal availability. The measure itself is then based on the distance or travel time to a single central business district, where all employment in a city is located ( Ia-cono and Levinson(2011)). This research will apply the same approach, although on a much larger scale. Using employment data for each zip code in the Netherlands, multiple accessibility measures are constructed for all 15,000 neighbourhoods in the Netherlands. As the main ques-tion of this paper is to asses and compare the empirical quality of the accessibility measures, the constructed measures will be placed in an empirical framework. This framework will be a house pricing model for all the aforementioned neighbourhoods. In this model, the estimation of prices per square meter will be performed using additional control variables such as crime rates and population density. The main question of this paper will then be answered by analyz-ing the predictive advantages of more complex accessibility measures, such as those includanalyz-ing net travel times, over benchmark accessibility models such as the number of jobs available in the zip code in which the neighbourhood lies.

In particular, the accessibility measures that will be compared in this paper vary in complex-ity. As a benchmark measure, the number of jobs available in a certain zip code in which the neighbourhood of interest lies will be used. A more complex measure is one that sums the num-ber of jobs that can be reached within 30 minutes by making use of public transport. An even more complex extension of this approach is to take a weighted sum of jobs that can be reached from the neighbourhood of interest. Here, the weighting of available jobs is done based on an

(8)

CHAPTER 1. INTRODUCTION 3

empirical function, which captures the willingness of the Dutch working population to travel a certain number of minutes to those respective jobs. This latter measure is used in a recent paper byHoogendoorn et al.(2016), whereas the former measure that sums the number of jobs within a fixed number of minutes from the origin is used by, for example,Wachs and Kumagai

(1973). Although it is fair to assume that the more complex measures capture more information about the actual accessibility of a neighbourhood than the simpler ones do, there has yet to be an empirical application of these measures to proof such a claim, which defines the goal of this research.

The rest of this paper is structured as follows. Chapter 2discusses the theoretical background of the used accessibility measures and house pricing models. Chapter 3shows in more detail the underlying models of the accessibility measures and a more detailed version of the house pricing models. Chapter 4then describes the data used in these models, and is followed by the empirical results inChapter 5. This paper concludes with a short conclusion and discussion in

(9)

Chapter 2 Theoretical Background

2.1 Housing Prices and Accessibility

In the comprehensive Handbook of regional and urban economics,Brueckner(1987) links pre-vious studies byAlonso(1964),Mills(1967) andMuth(1969) by their key observation that het-erogeneity in commuting and travel costs are balanced by differences in the cost of living space. Therefore, in a model that assumes a homogeneous unit cost for commuting trips, a positive relationship between housing prices and a measure of accessibility is expected. The models used in these papers focus on the dynamics within cities, noting implications such as the cen-tral city-suburban building height differential. By this implication, the clustering in economic centers can be explained. This clustering is still relevant today, and has been assessed more re-cently in a publication by the Dutch Bureau for Economic Policy Analysis (CPB). This extensive study byDe Groot et al.(2010) finds average land prices to be 200 times more expensive in Dutch city centers than in rural areas in the North of The Netherlands.

Extending the within-city models described byBrueckner(1987) to a larger scale,Hoogendoorn et al.(2016) describe the influences of improved accessibility on housing prices in the province of Zeeland in The Netherlands. By means of a natural experiment, they find the elasticity be-tween housing prices and accessibility to be 0.8. The researchers accredit this result partly to the anticipation effect, in which the expectation of better accessibility prematurely leads to higher prices on the housing market. The main result is also supported by studies such asGibbons

(10)

CHAPTER 2. THEORETICAL BACKGROUND 5

Figure 2.1: Percentage of residents with higher education

and Machin(2005) andArmstrong and Rodriguez(2006), who find that improved accessibility contributes to a rise in housing prices. In these papers, accessibility is defined by proximity to (commuter) rail stations, which accounts largely to so-called economical accessibility.Teulings et al.(2014) state that economical accessibility implies proximity to jobs and that accessibility therefore increases as more jobs can be reached within a certain time or distance. As the num-ber of job opportunities increases with accessibility, employees can choose from more firms that might suit their interests and skills. This can in turn increase productivity and therefore poten-tial wage, which leads to an increase in demand for accessible housing. Teulings et al.(2014) also find that this demand is higher for higher educated workers, as the value of their time is higher and a longer commute is therefore more undesirable. This result is in line with findings by Statistics Netherlands, who find the percentage of higher educated people to be larger in large cities, as can be seen inFigure 2.1.

(11)

Summarizing these results, a direct positive relationship is expected between a measure of (eco-nomic) accessibility and housing prices in a region. Defining accessibility can however be a challenge, as there seems to be little consensus in previous literature. As mentioned earlier, Gib-bons and Machin(2005) andArmstrong and Rodriguez(2006) define accessibility as proximity to (commuter) railway stations, whereas papers such as that ofBenenson et al.(2011) make use of more advanced measures including GPS data. Although fundamentally different, both these measures are examples of what has been defined as location-based (economical) accessibility measures byGeurs and Van Wee(2004). In their comprehensive overview, multiple forms of accessibility are defined and discussed, with usability being the central topic. Location-based accessibility measures are defined as measures that analyze accessibility at locations, describing the level of accessibility to spatially distributed activities. Location-based accessibility measures are most suited when comparing accessibility between multiple locations, such as neighbour-hoods. It is therefore this class of accessibility measures that will be used in the house pricing models, which in turn allow us to analyze and compare the accessibility measures by means of their predictive performance.

(12)

2.2 Construction of an Accessibility Measure

With the popularity of apps such as Google’s Directions on the rise, commuters can get an easy and clear overview of their travel possibilities. Expressing the cost of travelling from A to B in distance measures such as kilometers has become unpractical, as for each hour of the day the duration of the requested journey can be calculated in a measure of time. By making use of up-to-date traffic data and public transport timetables, the exact and optimal travel time can be found and used in the decision making process with respect to travel mode. These travel times can however also be used in defining an accessibility measure for a location of interest, by cal-culating the travel times to multiple destinations, such as economic masses. In equation (1) of their paper,Geurs and Van Wee(2004) define a general accessibility measure by means of such masses as: Ai= n X j =1 Dje−βci j (2.1)

InEquation 2.1, the accessibility of zone i is denoted by Ai, and it is equal to a weighted sum

of all (economic) opportunities D that can be reached from zone i . These opportunities are then multiplied with a weighting function with parameters measuring the travel cost (ci j) and

its sensitivity parameterβ. In their general formula,Geurs and Van Wee(2004) chose a negative exponential function as a weighting function, as this function is closely tied to travel behavior theory. Adaptations of this general formula are however also used in practice. For example,

Gutiérrez et al.(2010) make use of a direct inverse relation between travel time and accessibility, whereasHoogendoorn et al.(2016) make use of a country-specific empirical weighting function, which is also included in this paper.

Making use of the often proved relationship between housing prices and accessibility, see for example the comprehensive overview byBrueckner(1987), we will make use of house pricing models to compare the performance of the different accessibility measures. As the buying side of the housing market consists mainly of individuals, the economic masses in the used accessi-bility measure should satisfy the interest of individuals, rather than companies. Where compa-nies would define economic mass as the potential sales market at a certain location, individuals

(13)

Figure 2.2: Willingness to travelτ minutes to job

might choose their location strategically with respect to the number of jobs accessible from that location. Geurs and Van Wee(2004) name proximity to jobs as the most commonly used mea-sure for accessibility and also point out that this meamea-sure works well as a social indicator, show-ing the availability of social and economic opportunities for individuals. Furthermore,De Groot et al.(2010) note that the strategic choice of location is especially relevant for dual earning fam-ilies, which traditionally make up for a significant part of the housing market, emphasizing the importance of accessibility in this market.

In their paper concerning a natural accessibility experiment, Hoogendoorn et al.(2016) also define accessibility as a function of the number of jobs accessible. For their measure they make use of an empirical weighting functionconstructed by Marlet and van Woerkens(2007). This function measures the percentage of the Dutch working population that is willing to travel τ minutes to a job. Including this weighting function, the general formula for an accessibility measure in (2.1) simplifies to:

Ai= n

X

j =1

(14)

where the number of jobs available at a location is directly weighted with the time it takes to travel to this location. This weighting makes this accessibility measure far more complex than similar measures that only consider the number of jobs that can be reached within a certain travel time, which have also been used often in practice. For example, in their practical overview of accessibility measures,Handy and Niemeier(1997) note the frequent use of a measure that sums the number of jobs that can be accessed within a fixed number of minutes from a given location. Leading examples of this approach are the papers byWachs and Kumagai(1973) and

Sherman et al.(1974).

The third and most simple accessibility measure that will be considered in this research is one that only takes into consideration the number of jobs available at the location of interest itself. This measure is not named inHandy and Niemeier(1997) as a cumulative opportunity measure, but the choice for this simpler measure can be motivated by the fact that it is still unknown if the more complex accessibility measures hold more information about the actual accessibility of a given location. Especially when concerning economical accessibility, there has yet to be found an empirical proof for the superiority of the more complex accessibility measures over simpler ones.

This research therefore focuses on constructing the three aforementioned accessibility mea-sures for every neighbourhood in The Netherlands. These meamea-sures will then in turn be used in prediction models for housing prices in those neighbourhoods. The characteristics of these house pricing models are discussed in the following section.

(15)

2.3 House Pricing Models

In the previous two sections, the relationship between housing prices and accessibility has been thoroughly examined. Although the importance of location characteristics on housing prices has been discussed intensively in older literature, the influence of accessibility is also measured in more recent studies. The housing models used in this paper will therefore build on the house pricing models used in more recent literature.

2.3.1 Linear Pricing Models

Aforementioned papers by for exampleGibbons and Machin(2005),Armstrong and Rodriguez

(2006),De Groot et al.(2010),Iacono and Levinson(2011) andHoogendoorn et al.(2016) make use of (log) linear models to model the housing prices in the area of interest. With the exception of Hoogendoorn et al. (2016), all studies make use of cross sectional data. Generalizing the models used in these papers leads to the simplified equation1:

pi = α + Aiβ + Xiγ + ²i, (2.3)

where pi is the price of house i or the average housing price in region i . The corresponding

er-ror term is denoted by²i, which is expected to be independent and identically distributed with

mean zero. The explanatory variables are captured in matrix Xi and typically contain property

and neighbourhood characteristics. Special attention goes out to Ai, which is an accessibility

measure of some sort.

When making use of individual housing price data, the matrix consisting of explanatory vari-ables contains a number of lot-specific varivari-ables such as size and number of bathrooms. These types of variables are often used in hedonic housing formulas, but are not always available. In Table 5.1 of their paper,De Groot et al.(2010) overcome the problem of missing lot-specific data by making use of housing prices per square meter. This data is available for every neighbour-hood in The Netherlands and will also be used in this research.

1_{Note that}_{Gibbons and Machin}₍₂₀₀₅_{) and}_{Hoogendoorn et al.}₍₂₀₁₆_{) make use of a log-log specification for the}

(16)

Estimation of the model specified in (2.3) is in the previously mentioned papers (without ex-ception) done by means of a linear regression analysis, which includes fixed effects if the data includes some time varying component. Using a linear regression model makes for a straight-forward interpretation, but it imposes some serious restrictions on the model. For example, non-linearity’s in the model, such as that of the age of the house (as found by Grether and Mieszkowski(1974)), need to be specified beforehand. A possible solution to this problem is using the so-called lasso estimation on an extensive linear model including interactions and squared terms. The lasso method, as proposed by Tibshirani (1996), minimizes the sum of squared residuals subject to the sum of the absolute value of the estimated coefficients being less than a constant c.

ˆ

βL = arg_βminβ{ (y − Xβ)T(y − Xβ) subject toX i

| βi|≤ c} (2.4)

The constraint imposed by the constant c now forces some of the coefficients to be zero. A more general expression, however, makes use of the tuning parameterλ.

ˆ βL = arg_βminβ{ (y − Xβ)T(y − Xβ) + λ p X i =1 | βi| } (2.5)

The size of the parameterλ in (2.5) now determines the strength of the constraint, with larger values forλ leading to more coefficients being zero.Figure 2.3out ofJames et al.(2013) graphi-cally shows the working of the lasso constraint function when only two regressorsβ1andβ2are considered. The size of the blue constraint function directly relates to the size of the constant

c in (2.4) and has an inverse relationship with the value ofλ in (2.5). The intersection between

the constraint function and the red contour lines of the residual squared errors, yields a feasible solution given the restriction in (2.5). Smaller values for lambda therefore permit more possible solutions in which less coefficients are zero.

Despite the many possible approaches of estimating the linear model, no obvious predictive advantage has been found in favor of the linear model when being compared with newer meth-ods of estimation such as artificial neural networks (ANNs) or random decision forests. In fact,

(17)

Figure 2.3: Constraint Function of the Lasso Regressor

multiple papers such asNguyen and Cripps(2001) andPeterson and Flanagan(2009) find the converse to be true, with ANNs generating significantly lower dollar pricing errors. Especially in models that make use of a large number of dummy variables, such as housing models, ANN perform significantly better than linear models (Peterson and Flanagan(2009)).

2.3.2 Neural Network Pricing Models

Because the usage of artificial neural networks in house pricing models can lead to significant better performances, ANNs are also estimated in this paper. These neural networks were origi-nally designed to imitate the human learning process and the terminology therefore resembles that of the human brain. Each network consists of layers, which in turn consist of neurons. The more layers and neurons in the network, the more complex it becomes.

Typically, the network consists of an input layer, an output layer and one or more layers in be-tween, the so called hidden layers. The ANN models make use of the same input and output data as a linear regression model, but it does not have a fixed functional form. The form depends, among other things, on the number of neurons in each hidden layer and is therefore seldom linear. The number of neurons and layers is also not fixed, but can be chosen arbitrarily. Each

(18)

Figure 2.4: Example of an ANN

hidden layer contains two processes: the transformation function and the weighed summation function, linking the input to the output data (Haykin(1998)). Each neuron is trained to adjust the weight and the strength of the connections in the network by means of propagating errors back through the network (back propagation). The aim is then to optimize the sum of squared errors between the output of the network and the dependent variable, or target value (Peterson and Flanagan(2009)).Figure 2.4shows an example of a simple neural network with one hidden layer.

Empirical papers that estimate housing prices by means of an ANN, such as those byPeterson and Flanagan(2009) andSelim(2009), often make use of a fixed number of layers and neurons in their model. This paper makes, however, use of a trial-and-error based method to determine the optimal number of nodes in a network, which can be executed using the R-package nnet byVenables and Ripley(2002). Using this methodology, multiple models containing a different number of neurons are estimated and later compared by means of a validation set. The possible number of neurons considered in the estimated models are chosen within a bandwidth of about one third and two thirds of the number of variables used. The number of neurons that leads to the lowest MSE in the validation set is now used to train the final model, for which the complete training set is used. This model is used in the eventual comparison with the linear models and

(19)

the random decision forests.

This comparison is done based on new data, which is stored in the test set. The explanatory variables in this test set are now used as the new input layer of the optimal neural network cho-sen before. The output values that this network gives can now be compared with the actual target values in the test set, making it possible to calculate the prediction errors. The compar-ison of models is done based on these errors, as the coefficients itself have no direct interpre-tation in an ANN (Haykin(1998)). The main focus in this comparison will be on the influence of the three accessibility measures, which can be observed from the predictive accuracy of the used model. This predictive accuracy is measured by means of the MSE and provides us with evidence regarding the information contained in the different accessibility measures.

2.3.3 Decision Tree Pricing Models

The third and final method of estimation used in this research, is estimation by means of de-cision trees. This method has been around longer than the artificial neural networks, but has drawn some renewed attention following the publication byBreiman(2001). In his paper,Breiman

(2001) presents a method of estimation that combines multiple tree predictors in such a way that each tree depends on randomly drawn values. Because of the random nature of all calculated individual decision trees, the resulting estimate is called a random forest.

The basis of a random forest lies at the aggregation of basic decision trees. A standard deci-sion tree simply estimates a path that links each set of input variables x to the target variable y. An example of such a tree can be seen inFigure 2.5. This type of estimation yields an intuitive result, but tends to overfit the data, which makes for a relatively poor predictive performance of the model.

A random forest, however, averages over a large collection of K decision trees h(x;θk), where

allθk are independent and identically distributed random vectors. Each tree has its own

ran-dom vector, which not only ensures a different bootstrap sample of the training data, but also subsets the number of predictors available at each node. In a classical decision tree, each node

(20)

Figure 2.5: Example of a Decision Tree

is split using the best split among all variables. For a tree in a random forest, however, each node is split using the best split among a random subset of all possible variables (Liaw et al.(2002)). In short, each tree in a random forest is estimated using a random subset of the data and each node is split using a subset of all possible explanatory variables. Combining these trees in an equally weighted manner yields the random forest prediction.

¯ h(x) = 1 K K X k=1 h(x;θ_k) (2.6)

Using this definition, one can proof the superior accuracy of a random forest estimation. The following proof is based on that ofSegal(2004).

Using the Law of Large numbers, we find that

(21)

where the right hand side of (2.5) is equal to the prediction error of the random forest, denoted by PE∗_f. If we now define the average prediction error of an individual tree h(x;θ) to be

PE_t∗= EθEX,Y(Y − h(X;θ))2 (2.8)

we find that

PE∗_f ≤ ¯ρPE∗t (2.9)

Here, ¯ρ is an expression for the weighted correlation between Y −h(X;θ) and Y −h(X;θ0), withθ andθ0two independent random draws. Breiman(2001) now states that the accuracy improves as ¯ρ declines, and proves the superiority of a random forest over a single decision tree by in-cluding his own emprical research.Liu et al.(2013) even find that random forests compare quite favorably when compared with support vector machines (SVMs) and artificial neural networks when applied to classification data.

To exploit these possible advantages, random forest estimation is also included in this research. Together with artificial neural networks and the two linear models, three different estimation methods are applied to the data. The methodology of each of these methods is discussed in more detail in the next chapter.

(22)

Chapter 3 The Models

As mentioned in the previous chapter, three different accessibility measures are constructed for each of the more than 15.000 quarters and neighbourhoods in The Netherlands. The construc-tion of these measures is discussed first inSection 3.1. Subsequently, these measures are used in different house pricing models, which will be described inSection 3.2. These different models are then compared using the prediction errors, which can be tested against each other using a test proposed byDiebold and Mariano(1995), which is described inSection 3.3.

3.1 Modelling Accessibility

The simplest accessibility measure that is used in this research is the so-called local accessibility measure. This measure only counts the number of jobs available in the zip code area in which the respective neighbourhood lies. This measure has some obvious shortcomings, but serves as a benchmark in the rest of this paper.

The next two accessibility measures of interest make direct use of the public transport travel times between all zip codes in The Netherlands and share the same general formula ( Hoogen-doorn et al.(2016)): Ai = n X j =1 EjF (τi j) (3.1) 17

(23)

CHAPTER 3. THE MODELS 18

Here, the accessibility measure Ai is the sum of all jobs Ej that can be reached from

neighbour-hood i , weighted by the function F (τi j), whereτi j is the travel time between neighbourhood i

and the available jobs at location j .

The most commonly used accessibility measure sums the number of jobs that (public) trans-portation can reach within half an hour from the respective neighbourhood, making F (·) an indicator function that is equal to 1 ifτi j/l eq30 and 0 otherwise. The third and most advanced

accessibility measure makes use of all jobs that can be reached from the respective neighbour-hood. The available jobs are, however, weighted by anempirical weighting function. This func-tion is a continuous generalized Gaussian decay funcfunc-tion and takes into account travel times up to 90 minutes.

The public transport travel times themselves are obtained for all zip code areas in The Nether-lands. In order to construct the accessibility measures, we need travel times from every zip code area to every other zip code area in The Netherlands. This set of travel times is often referred to as an origin-destination matrix (O-D matrix) and can be obtained by sending travel requests to services like Google’s Directions.

These travel requests need to have both an existing origin and destination within the respec-tive zip code areas. These points are chosen as the centers of gravity of the respecrespec-tive zip code areas, which means that in most cases the travel time includes some walking time from the start-ing point of the journey to the closest mode of public transportation. This walkstart-ing is done with a speed of 5 km/h and happens both at the start and at the end of the journey.

After walking, the actual public transport journey begins, which can be calculated by using timetables of all available transportation companies. Services like Google’s Directions have these timetables built in, but unfortunately, most third-party applications have a limit on the amount of travel requests that can be made. Therefore, this research makes use of an open source trip planner, called the Open Trip Planner (OTP). It is possible to run this OTP on a local server, using self-supplied GTFS data, making it possible to send as many travel requests as one needs.

(24)

Figure 3.1: Example of accessibility of two neighbourhoods

To have a fair comparison of the calculated travel times, all travel requests in the O-D matrix are made on January 10, 2017, which is a Tuesday. Also, the travel times for each origin-destination pair are requested on twelve different times between 7AM and 8AM, in order to obtain a clearer overview of the actual travel times. In total, a number of 384,000,000 travel requests have been made, which ultimately made it possible to construct an ’accessibility range’ for each neigh-bourhood in The Netherlands. See for exampleFigure 3.1, which compares the accessibility of a neighbourhood in the center of Utrecht and a neighbourhood well outside this city. In this example, there is an obvious difference in the number of other neighbourhoods that can be ac-cessed within 90 minutes, which is also reflected in their constructed accessibility measures1. These accessibility measures are constructed by linking the available jobs at the reachable loca-tions to their respective travel times, before summing their (weighted) product. For the number of jobs available at each location, a data set from the LISA organization (LISA(2017)) is used.

After the (weighted) summation of the number of jobs, we end up with numerical values for the three accessibility measures. The first accessibility measure only counts the jobs that are

1_{The weighted accessibility of the respective zip code in Utrecht (3532) is about 4 times higher than the one in}

(25)

available in the zip code area in which the neighbourhood lies and will be referred to as the local accessibility measure. The second measure includes all jobs that can be reached within 30 min-utes and is referred to as the within30 accessibility measure. The third and final measure makes use of anempirical weighting functionand will be referred to as weighted. These three measures will now be included in different house pricing models, as will be discussed in the next section.

(26)

3.2 Modelling House Prices

3.2.1 Linear Pricing Models

Following the example ofGibbons and Machin(2005),Armstrong and Rodriguez(2006),De Groot et al.(2010),Iacono and Levinson(2011) andHoogendoorn et al.(2016), housing prices will first be modeled by means of a linear model. This model is estimated using OLS, which makes the in-terpretation of the model straight-forward. The model will be estimated using one of the three calculated accessibility measures and a set of control variables, which gives us the following three equations:

pi= α1+ Alocal ,iβ1 + Xiγ1+ ²i ,1 (3.2)

pi= α2+ Awithin30,iβ2+ Xiγ2+ ²i ,2 (3.3)

pi = α3+ Aweighted,iβ3+ Xiγ3+ ²i ,3 (3.4)

In these three models, the only explanatory variable that differs is the accessibility measure A. The set of control variables X is held constant, as this set contains neighbourhood specific data. The estimated coefficients forα, β and γ differ, making it possible to compare the different ac-cessibility models by, among other things, their predictive performance.

Possible complications concerning the linear models mentioned above include omitted non-linear terms and heteroscedastic error terms. The occurrence of both phenomena can be tested using the RESET test byRamsey(1969) and a Breusch-Pagan test byBreusch and Pagan(1979), respectively. The Breusch-Pagan test tests for heteroscedasticity of the error terms, which oc-curs by means of an auxiliary regression on the squared error terms. The null hypothesis of this test states that the error terms are not heteroscedastic and rejection of this hypothesis therefore confirms heteroscedasticity (Heij et al.(2004)). If this is the case, possible solutions include the usage of robust standard errors, as suggested byWhite(1980).

The RESET test tests whether higher powers of the explanatory variables have any power in ex-plaining the dependent variable. The null hypothesis states that this is not the case and the

(27)

rejection of this hypothesis implies misspecification of the model. The rejection of the null hy-pothesis therefore does not directly imply that nonlinear terms are omitted, but merely states that the model is misspecified (Heij et al.(2004)).

A possible solution for this misspecification is the lasso estimation method, as suggested by

Tibshirani(1996). This method of estimation allows us to include all possible interaction terms and squared terms of the available explanatory variables in the extended regressor matrix XL.

The lasso method will then choose the variables used in the actual model, leaving the other co-efficients at zero. The number of variables that are actually included in the model depends on the tuning parameterλ, where more coefficients are forced to be zero as λ increases.

The lasso model is estimated for all three accessibility measures, which now not only occur in-dividually, but are also included in the used interaction and squared terms. The lasso method-ology yields the following linear models:

pi = α4+ XL,local,i β4+ ²i ,4 (3.5)

pi= α5+ XL,within30,i β5+ ²i ,5 (3.6)

pi= α6+ XL,weighted,i β6+ ²i ,6 (3.7)

In these three models, the dimensions ofβ4,β5andβ6, differ from the dimensions ofβ1,β2and

β3. The number of non-zero coefficients in the lasso estimation can however be lower than the total number of variables used in the previous estimation, depending on the value forλ. This value is determined using 10-fold cross validation on the test yet, where theλ that yields the lowest cross validated error is chosen in the eventual prediction model.

These prediction models are then compared to each other by means of their predictive per-formance on the test set. Previous to the actual estimation of the models, the data is split in two non-overlapping groups: the training set and the test set. The training set contains 80% of the data and is used to train the models. For the linear case, this means the regression coefficients

(28)

are estimated using this data. Using these coefficients, the explanatory variables in the test set can be used to construct a prediction for all dependent variables in the test set. By comparing the predicted values with the actual values, we are able to determine an expression for the pre-diction error. These errors then allow us to not only compare the performances of the different linear models, but also gives us the possibility to compare the predictive power of the linear model with that of the artificial neural networks and decision tree based models.

3.2.2 Neural Network Pricing Model

Following evidence fromHuang et al.(1994),Nguyen and Cripps(2001) andPeterson and Flana-gan(2009), the house pricing model is also estimated using Artificial Neural Networks (ANNs). For the estimation of this model, the R package nnet byVenables and Ripley (2002) is used. This package trains the model using back propagation of the errors, for which the default error function is the mean squared error (MSE). The output units are set to be linear, as our aim is to predict a numerical value.

The estimation itself resembles that of a black box, as studying the model’s structure will not give us any insights on the structure of the function being approximated. Also, the estimated coefficients within the network have no direct interpretation, as they have in, for example, the linear case. These issues are however not of great importance, as we aim to compare the models by their predictive abilities, which will be done by focusing on the MSE.

In the network itself, we fix the number of layers at one and allow the optimization of the neu-ral network to vary the number of nodes in this layer. As a rule of thumb, one often chooses the number of nodes to be within the range of one third and two thirds of the number of input variables (Haykin(1998)). We will also loosely follow this methodology, as it prevents us from overfitting.

To determine the optimal neural network, we again use the same 80/20 data split as before in the linear case. We will however also split the training set into a CV-training set and a validation set, which do not overlap. Here, the CV-training set contains 75% of the data of the old training

(29)

set and hence 60% of the total available data. The other 25% of the old training set is called the validation set , which contains 20% of the total available data.

The networks are now trained on the CV-training set, before being evaluated using the validation set. The network with the lowest MSE in the validation set is chosen to be optimal and will again be trained using the full training set, which is also used in the linear models. This methodology is repeated for all three models (one for every accessibility measure), implying that the models possibly vary in the number of nodes used.

The three models are then applied to the test set, in which the explanatory variables now serve as a new input layer for the neural networks. Each network then predicts the output variable us-ing their previously determined weights and connections, yieldus-ing different estimations for all three models. Again, the predictive performances of the models are compared by determining the error between the predicted values and the actual housing prices in the test set. The expres-sion used for the error is, just as for the other two methods of estimation, equal to the mean squared error.

3.2.3 Decision Tree Pricing Model

The final estimation method used in this research is decision tree based estimation. As stated in the previous chapter, the random forest estimation method makes use of an aggregation of a large number of decision trees. In this research, the R package randomForest byLiaw et al.(2002) is used to estimate the random forest models. This package allows us to, among other things, vary the number of used explanatory variables at each node of the tree. As a rule of thumb, one often chooses this number to be one third of the total number of explanatory variables. In this research, that would imply 6 explanatory variables used at each node. Liaw et al.(2002) however suggest to cross validate this parameter by also trying half the default and double the default. This methodology is also used in this research, as using 3, 9 or 12 variables at each node is also allowed. The four aforementioned model options are now trained on the full training set. This can be done without overfitting the model, as this is often no serious issue in random forest estimation (Breiman(2001)). The model with the lowest error in the training set is now chosen

(30)

as the optimal model and is used for predicting the housing prices in the test set. The prediction errors are then used to compare all previously described models. This can be done by means of a test proposed byDiebold and Mariano(1995), which is described in the next section.

(31)

3.3 Comparing the Models

As the goal of a prediction model is accuracy, prediction models are best compared by their predictive performance. This performance can be expressed in terms of the model’s prediction errors, which follow from the difference between the predicted value ˆy and the actual dependent

variable y. To make the comparison between multiple models easier, a loss function is often applied to the prediction errors. A simple loss function is the mean squared error, which is defined as follows: M SE = 1 n n X i =1 ( ˆyi− yi)2 (3.8)

Although this measure makes for an easy comparison between multiple predictions, it does not give any indication about the significance of the observed difference. For this purpose,Diebold and Mariano (1995) propose a test using the complete sequence of prediction errors of each model. The resulting test statistic, which is derived below, is asymptotically N(0,1) distributed and can hence be compared with its critical value, determining the significance of the difference in forecast errors.

First off, the forecast error at place or time t of model i can be defined as ei t = ˆyi t− yt. The

associated loss of this error is often expressed as a function g (·) of the actual error, which de-fines the loss differential between two predictions to be:

dt = g (e1t) − g (e2t) (3.9)

The null hypothesis of the test states that the forecasts do not differ, hence resulting in an ex-pected loss differential of zero for all t . This statement corresponds to the claim that the pop-ulation meanµ of the loss differential is 0 (Diebold and Mariano (1995)). If we now assume the loss differential series to be covariance stationary and of short memory, one can derive the asymptotic distribution of the sample mean to be:

p

(32)

where ¯d is the sample mean of the aforementioned loss differential and

fd(0) = 1 2π ∞ X τ=−∞γd(τ) (3.11)

the spectral density of the loss differential at frequency 0 (Diebold and Mariano(1995)). This spectral density contains the quantityγd(τ), which is autocovariance of the loss differential at

displacementτ.

A consistent estimate of fd(0) is given by:

ˆ fd(0) = 1 2π Ã ˆ γd(0) + 2 h−1 X τ=1 ˆ γd(τ) ! , (3.12)

with h the number of steps ahead in the forecast and

ˆ γd(τ) = 1 T T X t =|τ|+1 (dt− ¯d )(dt −|τ|− ¯d ) (3.13)

a consistent and symmetric estimate ofγd(τ). SeeDiebold and Mariano(1995) for a more

de-tailed derivation of above quantities.

Using (3.12) in (3.10) and the null hypothesisµ = 0, we arrive at the quantity of interest:

D M = ¯ d q 2π ˆfd(0) T d −→ N (0, 1) (3.14)

Using (3.14), a two-sided Diebold-Mariano test rejects the null hypothesis if | DM |> zα/2.

Harvey et al. (1997) claim, however, that an improvement of the small-sample properties can be achieved by making a small bias correction in the Diebold-Mariano test statistic in (3.14). The correction applied to this statistic is the following:

H D M = s T + 1 − 2h +_T1h(h − 1) T ¯ d q 2π ˆfd(0) T (3.15)

(33)

Besides this bias correction,Harvey et al.(1997) also suggest that the critical values used in the Diebold-Mariano test are replaced by critical values from the Student’s t distribution with T − 1 degrees of freedom. This methodology would be correct for the one-step ahead predictor if the loss differentials dtwere to be normally distributed.Harvey et al.(1997) claim that the normality

assumption will not hold in general, but that the Student t critical values would still be more ap-propriate. It is therefore this methodology that is used when comparing the predictions made by two different models. In this paper, a two sided test is performed on the null hypothesis, which states that the population mean of the loss differential, denoted byµ, is zero. In the context of this paper, the null hypothesis states that the predictions made by two different models do not differ significantly from each other, while the alternative hypothesis merely states that they do differ significantly. We can therefore not conclude that one model is better than the other if the null hypothesis is rejected. However, combining the test with an expression for model perfor-mance, such as the MSE, does give us insight in the possible superiority of one model over the other.

(34)

Chapter 4 Data Description

Each model mentioned in the previous chapter makes use of the same data, of which the char-acteristics are presented in this chapter. First, the respective sources of the data are named. Then, the variable names are discussed, before presenting some summarizing statistics.

The Netherlands counts 15,446 quarters and neighbourhoods, with the number of inhabitants ranging from 100 to 27,650. For all quarters and neighbourhoods, Statistics Netherlands (CBS) publishes yearly information regarding demographics and the housing situation. The most re-cent publication, Kerncijfers Wijken en Buurten 2015, contains 102 variables, of which 29 can be used in this paper. A complete overview can be found in the CBS-supplied table in the Ap-pendix, which also contains a brief explanation of the variables in Dutch.

Other CBS publications that include data on quarter and neighbourhood level can be com-bined by means of the neighbourhood code. Additional explanatory variables that are obtained this way include crime rates, which are found in the publication Geregistreerde criminaliteit per

gemeente, wijk en buurt (2010-2015). For the average housing price per square meter, the

publi-cation Gemiddelde WOZ-waarde per vierkante meter, 2012 en 2015 is used.

After combining all data sources, 6329 observations are omitted, as they cannot be used. A large part of these omitted observations are observations belonging to quarters. This is due to the fact that quarters and neighbourhoods are not mutually exclusive, with one quarter

(35)

CHAPTER 4. DATA DESCRIPTION 30

ing one or multiple neighbourhoods. Other observations that were removed contained one or more missing values. There was no obvious pattern discovered as to which observations were excluded, but data belonging to some smaller neighbourhoods were incomplete due to privacy reasons.

For the remaining observations, a total of 37 variables can be used, including the three con-structed accessibility measures and some location dummies for Amsterdam, Rotterdam and The Hague. The list of variables can be found in Table 4.1. Now follows a short explanation of each variable in this table.

Total_Inhabitants measures the total number of inhabitants in a neighbourhood. This

vari-able is used to transform absolute values to percentages, in particular for the age varivari-ables

Perc_0_14, Perc_15_24, Perc_24_44, Perc_45_64 and Perc_65_00. These variables measure the

percentage of the inhabitants that are between, for example, zero and fourteen years old. The last variable concerning inhabitants is Perc_NonWestern, which measures the fraction of the inhabitants that is of non-Western origin.

The variable Total_Households is used to calculate the percentage of households that consist of one person as captured by Perc_1PersonHouseholds. The Average_HouseholdSize measures the average number of people living in a household in the respective neighbourhood.

Pop-Density measures the number of people living on one km2 and is calculated by dividing the

Total_Inhabitants by the area of the neighbourhood.

The total number of houses in a neighbourhood is captured in the variable Total_Houses. This variable is also used when calculating Perc_FamilyHomes, Perc_Empty, Perc_OwnerOcc and

Perc_Younger2000. These variables contain the percentage of houses which are respectively

family homes, empty, owner occupied or built later than the year 2000.

Average_Gas measures the average gas consumption per year in m3and Average_Income

(36)

measures the number of companies that are present in the respective neighbourhood.

The variables Dist_HP, Dist_SM, Dist_DC and Dist_School measure (in kilometers) the aver-age distance of all inhabitants to services like hospitals, supermarkets, day cares and primary schools. Total_Schools measures the average number of schools that can be reached by the in-habitants within a range of 3 kilometers and is therefore not necessarily an integer.

The total acres of water and land in a neighbourhood are captured by Total_AcresWater and

Total_AcresLand. Their ratio, which measures the number of acres of water for each acre of

land, is called WaterLandRatio. Zipcode is a numerical value that states the most frequently oc-curring zip code in a neighbourhood and is used to determine whether the neighbourhood lies in one of the three major cities in The Netherlands. This information is captured in the dum-mies In_Amsterdam, In_Rotterdam and In_TheHague.

The neighbourhood’s level of urbanity is expressed on a one to five scale by Statistics Nether-lands and is named Urbanity. Crime rates are defined as the number of vandalism and vio-lence offenses per 1000 inhabitants and are available under the name Crimerate. Special at-tention goes out to the three constructed accessibility measures. WeightedAM takes a weighted sum of the number of jobs that can be reached from the respective neighbourhood, whereas

Within30AM only sums the jobs that can be reached within 30 minutes. Benchmark measure LocalAM is even more simplified, as it only takes into account the number of jobs in the zip code

area in which the neighbourhood lies.

The dependent variable in the house pricing models is Houseprice_m2. This variable contains the average housing price per m2and is derived from the WOZ-waarde in the respective neigh-bourhood. The WOZ-waarde is an estimation of the actual market value, made by the munici-pality in which the neighbourhood lies. The value is often calculated as a weighted average of numerous past home sales in the same area and is based on full and unencumbered property ownership. A histogram of this variable can be found inFigure B.1 of the Appendix.

(37)

Table 4.1: Descriptive Statistics of Variables Used

Statistic N Mean St. Dev. Min Max

Total_Inhabitants 9,117 1,759.137 2,100.169 100 27,650 Perc_0_14 9,117 0.163 0.049 0.000 0.416 Perc_15_24 9,117 0.120 0.043 0.000 0.754 Perc_25_44 9,117 0.226 0.071 0.000 0.696 Perc_45_64 9,117 0.301 0.062 0.016 0.543 Perc_65_00 9,117 0.191 0.088 0.000 0.951 Perc_NonWestern 9,117 0.069 0.097 0.000 0.882 Total_Households 9,117 800.337 1,035.245 30 13,935 Perc_1PersonHouseholds 9,117 0.312 0.134 0.000 0.944 Average_HouseholdSize 9,117 2.325 0.378 1.100 4.100 PopDensity 9,117 3,258.101 3,444.160 4 28,599 Total_Houses 9,117 794.250 998.972 27 14,326 Perc_FamilyHomes 9,117 0.778 0.255 0.000 1.000 Perc_Empty 9,117 0.056 0.060 0.000 0.870 Perc_OwnerOcc 9,117 0.668 0.210 0.000 1.000 Perc_Younger2000 9,117 0.863 0.199 0.000 1.000 Average_Gas 9,117 1,579.873 519.719 10 5,050 Average_Income 9,117 24.148 5.653 7.700 84.400 Total_Companies 9,117 138.588 193.252 0 3,965 Dist_HP 9,117 1.485 1.277 0.100 9.600 Dist_SM 9,117 1.411 1.264 0.100 9.300 Dist_DC 9,117 1.341 1.317 0.100 13.100 Dist_School 9,117 0.959 0.774 0.100 9.400 Total_Schools 9,117 7.940 7.511 0.000 62.400 Total_AcresLand 9,117 265.474 570.560 2 12,822 Total_AcresWater 9,117 8.698 37.040 0 1,033 WaterLandRatio 9,117 0.043 0.160 0.000 7.083 Zipcode 9,117 5,293.959 2,489.988 1,011 9,997 Urbanity 9,117 3.494 1.476 1 5 Crimerate 9,117 41.277 197.046 0.000 7,300.000 In_Amsterdam 9,117 0.010 0.098 0 1 In_Rotterdam 9,117 0.007 0.086 0 1 In_TheHague 9,117 0.011 0.106 0 1 WeightedAM 9,117 89,687.370 76,407.100 50 667,279 Within30AM 9,117 38,947.430 45,095.510 50 489,800 LocalAM 9,117 2,888.710 3,251.184 10 38,050 Houseprice_m2 9,117 1,839.799 596.914 500 26,050

(38)

Chapter 5 Results

This chapter presents the estimation results for each of the different models and methods. All models are eventually estimated using the same training set containing 80% of the data, which leaves 20% of the data to test the predictions made by the models. The predicted values are compared with the actual housing prices in the test set, resulting in an expression for the error of the prediction. In this paper, the mean squared error (MSE) and the Diebold-Mariano test are used to compare the different models and methods. As the partition of the data into a training, validation and training set is randomly done, this process is repeated five times, as is the esti-mation and the prediction. The results for each data partition are presented, as is the average error over all data partitions.

First off, the estimation results for the linear models are presented in Section 5.1. The coeffi-cients of the various variables are briefly discussed and the overall performance of the models is evaluated. InSection 5.2andSection 5.3, the estimation results of the artificial neural networks and random forest estimations are presented. Finally, in Section 5.4, the errors of the linear models, neural networks and random forests are compared and a short conclusion is drawn.

(39)

CHAPTER 5. RESULTS 34

5.1 Linear Estimation Results

The linear models are estimated by an ordinary least squares regression on the training set. This training set contains 7294 observations that are randomly drawn. To avoid an unlucky draw, this process is repeated five times, which yields five different estimation results and hence five differ-ent MSEs for the predictions made. The complete estimation results for each of the five differdiffer-ent data partitions can be found in theAppendix, but the result of a regression on the complete data set can be found below, inTable 5.1.

Although the main focus of this paper lies with the differences between the three accessibility models, the signs of some of the estimated coefficients are also noteworthy. Therefore, before the comparison of model performances, the coefficients of the full regression will be briefly dis-cussed.

One of the regression’s noteworthy results is the magnitude of the variable Perc_NonWestern. In all three models, this variable is highly significant and has a relatively large negative influence on the housing prices in a neighbourhood. In the model using the weighted accessibility mea-sure, for example, the housing price per m2declines with 8 euros for each additional percentage point of non-Western inhabitants. Also, lower housing prices are found in neighbourhoods con-taining a high percentage of owner occupied houses and houses built after the year 2000.

Coefficients with a noteworthy positive influence on the housing price are the average income per inhabitant, the water to land ratio and the average household size, as this is a proxy of the actual average house size. Also, houses located in the city center of Amsterdam are significantly more expensive than their counterparts in the rest of the country.

Our main interest lies with the accessibility measures and the differences between the three models. In each model, the accessibility measure is highly significant, with p-values well be-low the 1 percent level. The weighted and the within30 accessibility measure have an estimated coefficient of the same magnitude, whereas the coefficient of the local measure is roughly five

(40)

Table 5.1: Linear Estimation Result for All Data

Dependent variable:

Houseprice_m2

Weighted Within 30 minutes Local

Constant −1,657.516∗∗∗(122.833) −2,146.142∗∗∗(123.892) −2,338.558∗∗∗(122.721) Perc_NonWestern −802.517∗∗∗(77.293) −542.073∗∗∗(78.220) −430.388∗∗∗(78.169) Perc_1PersonHouseholds 1,637.991∗∗∗(101.065) 1,967.519∗∗∗(103.014) 2,118.546∗∗∗(102.468) Average_HouseholdSize 694.389∗∗∗(33.341) 826.062∗∗∗(33.562) 852.703∗∗∗(33.646) PopDensity 0.002 (0.002) 0.006∗∗(0.002) 0.008∗∗∗(0.003) Perc_Empty 522.141∗∗∗(89.801) 468.662∗∗∗(91.958) 434.852∗∗∗(92.362) Perc_OwnerOcc −410.020∗∗∗(44.336) −440.207∗∗∗(45.425) −422.384∗∗∗(45.639) Perc_Younger2000 −71.519∗∗(28.207) −24.271 (28.824) −19.207 (29.007) Average_Gas 0.070∗∗∗(0.016) 0.038∗∗(0.016) 0.032∗∗(0.016) Average_Income 48.388∗∗∗(1.229) 57.078∗∗∗(1.188) 59.285∗∗∗(1.169) Dist_HP 4.210 (4.844) 4.308 (4.966) 7.483 (4.993) Total_Schools 2.130 (1.332) 5.362∗∗∗(1.389) 8.980∗∗∗(1.338) WaterLandRatio 224.644∗∗∗(30.568) 249.827∗∗∗(31.298) 244.935∗∗∗(31.445) Urbanity 57.577∗∗∗(7.139) 54.093∗∗∗(7.318) 60.137∗∗∗(7.396) Crimerate 0.008 (0.025) 0.028 (0.025) 0.036 (0.025) In_Amsterdam 693.290∗∗∗(55.224) 748.723∗∗∗(60.788) 993.189∗∗∗(55.267) In_Rotterdam −327.088∗∗∗(58.809) −308.047∗∗∗(60.768) −213.190∗∗∗(60.319) In_TheHague 30.580 (52.708) −171.114∗∗∗(53.406) −272.774∗∗∗(52.535) WeightedAM 0.002∗∗∗(0.0001) Within30AM 0.002∗∗∗(0.0002) LocalAM 0.007∗∗∗(0.002) RESET 9.299193*** 19.86523*** 22.55873*** Breusch Pagan 54.498*** 55.037*** 55.445*** Observations 9,117 9,117 9,117 R2 0.422 0.394 0.388 Adjusted R2 0.421 0.393 0.387 Residual Std. Error 454.084 465.128 467.267 F Statistic 369.704∗∗∗ 328.639∗∗∗ 321.020∗∗∗ Note: ∗p<0.1;∗∗p<0.05;∗∗∗p<0.01

(41)

times larger. The difference in magnitude of the estimated coefficients can partly be explained by the average value of the accessibility measures (seeTable 4.1), but this argument does not hold in all cases.

An early conclusion regarding the predictive power of the three models can be drawn by means of the R2and the F statistic. Both the R2and the F statistic are highest for the weighted acces-sibility measure and lowest for the local measure. This pattern can also be found in the five estimations for the different data partitions, as can be seen in theAppendix.

Table 5.2: Mean Squared Errors for OLS Models

Partition 1 Partition 2 Partition 3 Partition 4 Partition 5 Average

LocalAM 143883 492080 155966 138859 143364 214830

Within30AM 142066 491090 153998 135909 140986 212810

WeightedAM 134580 480486 142544 126374 130811 202959

The above early conclusion is supported when we compare the actual prediction errors, which can be found inTable 5.2. The most complex accessibility measure that was constructed, the

WeightedAM, yields the lowest prediction error in all data partitions and hence has the

low-est average error. As to be expected, the simpllow-est accessibility measure, the LocalAM, yields the highest prediction error for all partitions and hence has the highest average error. Another interesting result is that the average error for the within30 accessibility measure lies closer to that of the local measure than to the error of the weighted measure. This result is surpris-ing, as the extra effort and computing power needed to construct the Within30AM instead of the LocalAM is much higher than the difference in effort between the WeightedAM and the

Within30AM. One would therefore suspect that the gap between the errors of the WeightedAM

and the Within30AM would be smaller than the gap between the Within30AM and LocalAM, but instead the converse is true.

The significance of the differences between the predictions made by the models can be tested by means of an adapted version of the Diebold-Mariano test (Diebold and Mariano(1995), Har-vey et al.(1997)). The test statistics of this test can be found inTable 5.3. Each column contains statistics of the tests that are executed using the errors of the models containing two respective

(42)

Table 5.3: Diebold-Mariano Test Statistics for OLS Models

LocalAM - Within30AM LocalAM - WeightedAM Within30AM - WeightedAM

Partition 1 1.9102146 3.966927* 4.150527*

Partition 2 0.7286697 4.065256* 5.703146*

Partition 3 2.1089411* 5.924284* 6.850922*

Partition 4 3.4769515* 5.698088* 5.483406*

Partition 5 2.5620020* 5.593839* 6.053765*

accessibility measures. For example, the first column tests the errors of the model containing the local accessibility measure against those of the model containing the within30 accessibility measure. Test statistics marked with an asterisk(*) reject the null hypothesis at the 5% signifi-cance level, concluding that both predictions differ significantly from each other.

The results in Table 5.3are partly inconclusive, as the predictions made by the models con-taining the LocalAM and the Within30AM do not differ significantly in the first two data parti-tions. In the other three data partitions, however, the predictions made by the models do differ significantly, with the model containing the Within30AM yielding a lower MSE. The other two columns contain test statistics concerning the WeightedAM model. The predictions made by the WeightedAM model do not only yield the lowest MSE, but its predictions also differ signifi-cantly from those made by the other models. We therefore conclude that the model containing the weighted accessibility measure is superior in the case of an ordinary least squares regression.

In spite of its simplicity, the OLS model does have some shortcomings. Analysing the Breusch-Pagan and RESET test statistics in Table 5.1and in the separate regressions in the Appendix, we find evidence for misspecification and heteroscedastic errors. The latter problem can be overcome by usingWhite(1980) standard errors, but this modification does not affect the pre-dictions made by the model and is therefore not discussed further. The misspecification and the possibility of missing non linear terms in the model is however a more severe threat to the performance of our models. In all data partitions, the null hypothesis of no misspecification is rejected and action is therefore warranted. A possible solution for the misspecification is apply-ing a lasso estimation on all the available data, includapply-ing interaction and squared terms.

(43)

Table 5.4: Mean Squared Errors for Lasso Models

Partition 1 Partition 2 Partition 3 Partition 4 Partition 5 Average

LocalAM 139127 488464 141057 132664 133739 207010

Within30AM 136640 487814 140900 131476 131517 205669

WeightedAM 129305 472689 127913 122612 119501 194404

Table 5.5: Diebold-Mariano Test Statistics for Lasso Models

LocalAM - Within30AM LocalAM - WeightedAM Within30AM - WeightedAM

Partition 1 1.3532018 2.295707* 2.060184*

Partition 2 0.3108321 2.974491* 3.671973*

Partition 3 0.1481557 4.937994* 6.244910*

Partition 4 1.1742656 4.301074* 4.721367*

Partition 5 1.5551890 4.568951* 5.581853*

The lasso model is trained on the training set using 10-fold cross validation, resulting in a λ that minimizes the cross validated error. Thisλ is then used to construct the eventual model that will be used to predict the housing prices in the test set. Despite the possible gains in accu-racy, direct interpretation of the lasso estimation itself is tedious. Variables that are not included in the optimal model are not necessarily worse than the variables that are selected by the lasso and we can therefore not conclude much based on this selection. We can however compare the different models by means of their prediction errors (Table 5.4)and the Diebold-Mariano test statistics (Table 5.5).

Doing so gives us the same conclusion as for the OLS estimations. In all data partitions, the MSE of the WeightedAM model is the lowest, which naturally results in it having the lowest average error as well. Furthermore, the Within30AM performs better than the LocalAM, although the difference between these errors is smaller than the difference in errors between the Within30AM and the WeightedAM. This observation is supported by the results of the Diebold-Mariano test, which concludes that there is no significant difference in the predictions made by models con-taining the local and the within30 accessibility measures. For the comparison of these two mod-els with the weighted accessibility measure, such a significant difference is found, confirming the superiority of linear models using this WeightedAM.

(44)

5.2 Neural Network Estimation Results

Just as for the lasso estimation in the previous section, neural networks cannot be compared based on their direct estimation output. There are many connections between the different layers and the interpretation of the coefficients belonging to these connections is unfortunately not as straight forward as in the linear case. A graphical representation often gives a good insight in the model structure, and examples of the estimated neural networks can therefore be found inFigure 5.1and in theAppendix.

Figure 5.1: Example of a Neural Network Structure Using WeightedAM

The main focus of this paper is however not to confirm the relationship between housing prices and accessibility, but to use the house pricing models to determine which accessibility measure holds the highest predictive power. We will therefore now focus on the prediction errors of the three models, which are displayed inTable 5.6.

Measuring accessibility : an empirical comparison of accessibility measures in house pricing models in The Netherlands