Non-global regression modelling

(1)

by Yunkai Huang

BMgmt, Yunnan University, 2002

BA (Honours), The University of Winnipeg, 2006 BSc, The University of Winnipeg, 2006

MA, Carleton University, 2007

A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of

DOCTOR OF PHILOSOPHY in the Department of Economics

 Yunkai Huang, 2016 University of Victoria

(2)

Supervisory Committee

Non-Global Regression Modelling by

Yunkai Huang

BMgmt, Yunnan University, 2002

BA (Honours), The University of Winnipeg, 2006 BSc, The University of Winnipeg, 2006

MA, Carleton University, 2007

Supervisory Committee

Dr. David E.A. Giles, Department of Economics

Supervisor

Dr. Judith A. Clarke, Department of Economics

Departmental Member

Dr. Farouk Nathoo, Department of Mathematics and Statistics

(3)

Abstract

Supervisory Committee

Dr. David E.A. Giles, Department of Economics

Supervisor

Dr. Judith A. Clarke, Department of Economics

Departmental Member

Dr. Farouk Nathoo, Department of Mathematics and Statistics

Outside Member

In this dissertation, a new non-global regression model - the partial linear threshold regression model (PLTRM) - is proposed. Various issues related to the PLTRM are discussed.

In the first main section of the dissertation (Chapter 2), we define what is meant by the term “non-global regression model”, and we provide a brief review of the current literature associated with such models. In particular, we focus on their advantages and disadvantages in terms of their statistical properties. Because there are some weaknesses in the existing non-global regression models, we propose the PLTRM. The PLTRM combines non-parametric modelling with the traditional threshold regression models (TRMs), and hence can be thought of as an extension of the later models. We verify the performance of the PLTRM through a series of Monte Carlo simulation experiments. These experiments use a simulated data set that exhibits partial linear and partial

nonlinear characteristics, and the PLTRM out-performs several competing parametric and non-parametric models in terms of the Mean Squared Error (MSE) of the within-sample fit.

(4)

In the second main section of this dissertation (Chapter 3), we propose a method of estimation for the PLTRM. This requires estimating the parameters of the parametric part of the model; estimating the threshold; and fitting the non-parametric component of the model. An “unbalanced penalized least squares” approach is used. This involves using restricted penalized regression spline and smoothing spline techniques for the

non-parametric component of the model; the least squares method for the linear

parametric part of the model; together with a search procedure to estimate the threshold value. This estimation procedure is discussed for three mutually exclusive situations, which are classified according to the way in which the two components of the PLTRM “join” at the threshold. Bootstrap sampling distributions of the estimators are provided using the parametric bootstrap technique. The various estimators appear to have good sampling properties in most of the situations that are considered. Inference issues such as hypothesis testing and confidence interval construction for the PLTRM are also

investigated.

In the third main section of the dissertation (Chapter 4), we illustrate the usefulness of the PLTRM, and the application of the proposed estimation methods, by modelling various real-world data sets. These examples demonstrate both the good statistical performance, and the great application potential, of the PLTRM.

(5)

List of Tables

Table 2-5.1 Risks of different models over 1,000 simulated samples ... 35

Table 3-4.1 Simulation studies for model (3-4.1) with n∈{33,65,129, 257, 513,1025}, 1∈{1.5,3} β , and α∈ −{ 0.5,0, 0.475} ... 78

Table 3-4.2 Risks reassessed over 1,000 samples... 83

Table 3-5.1 Selected percentiles of τ_b for the example ... 87

Table 3-5.2 Confidence intervals for

α

... 88

Table 4-4.1 Weight–height ratio and age for pre-school boys... 122

Table C.1 Normality test statistics for DGP1: α= −0.5,β₀=2,β₁=1.5,n=33 ... 145

Table C.2 Normality test statistics for DGP2: α= ,0 β₀=2,β₁=1.5,n=33 ... 146

Table C.3 Normality test statistics for DGP3: α=0.475,β₀=2,β₁=1.5,n=33 ... 147

Table C.4 Normality test statistics for DGP4: α= −0.5,β₀=2,β₁=3,n=33 ... 148

Table C.5 Normality test statistics for DGP5*: α= ,0 β₀=2,β₁=3,n=33 ... 149

Table C.6 Normality test statistics for DGP6: α=0.475,β₀=2,β₁=3,n=33 ... 150

(8)

Table C.32 Normality test statistics for DGP32: α= ,0 β₀=2,β₁=1.5,n=1025... 176

Table C.33 Normality test statistics for DGP33: α=0.475,β₀=2,β₁=1.5,n=1025 .... 177

Table D.1 Global land-ocean temperature index (C) (Anomaly with Base:1951-1980) 181 Table D.2 UK household net income and food expenditure (new pounds per week) .... 182

(9)

List of Figures

Figure 2-2.1 Weight-height ratio versus age for preschool boys: ... 13

Figure 2-2.2 Motorcycle-helmet impact data with a cubic smoothing spline... 15

Figure 2-3.1 Non-global model with discontinuous change ... 22

Figure 2-3.2 Non-global model with continuous change... 23

Figure 2-3.3 Hyperbolic tangent transition tanh(x a) b − for a=0 and b=1, 2, 3 ... 25

Figure 2-3.4 Non-global smooth transition model with s x( ) 0.5 0.5 tanh(x 1) b − = + ... 26

Figure 2-5.1 Scatter plot of a typical realization of 129 observations and the true model 32 Figure 2-5.2 Scatter plot and different fits: i true model-True (black); ii linear model-Linear (blue); iii cubic smoothing spline-SS (red); iv partial linear model-PL (purple); v segmented linear model with two break points-SL (brown); vi quadratic B-spline-QB (golden); vii partial linear threshold regression model-PLTRM (green) .... 32

Figure 2-5.3 Fits against true model ... 34

Figure 3-3.1 Types of join ... 53

Figure 3-4.1 Model (3-4.1) with α=0,β₁=1.5 and n=257 ... 79

Figure 3-4.2 Sampling distribution of α with DGP 20 ... 80ˆ Figure 3-4.3 Sampling distribution of β with DGP 20 ... 81ˆ₀ Figure 3-4.4 Sampling distribution of β with DGP 20 ... 81ˆ₁ Figure 3-5.1 Bootstrap distribution of τ_b for the example ... 86

Figure 3-5.2 Fitted PLTRM with confidence band ... 92

Figure 3-5.3 PLTRM fit and its first order derivatives ... 93

Figure 4-2.1 Plot of daily SCI and log (SCI) ... 100

Figure 4-2.2 Modelling the Trends for log (SCI Closing) i. linear model (black); ii. PLTRM(green) iii. Smoothing Splines (blue and red) (phi=.9 and phi=.4); ... 102

Figure 4-2.3 De-trending plots with different trend... 104

Figure 4-2.4 Trend prediction with different models... 105

Figure 4-2.5 Global Land-Ocean Temperature Index (1880-2014, annual) ... 107

Figure 4-2.6 Trends of Global Temperature ... 109

Figure 4-2.7 Variations of the Temperature Anomaly ... 109

Figure 4-2.8 Local Derivative of the PLTRM Trend ... 110

Figure 4-3.1 Estimated Engel curves; UK Family Expenditure Survey 1976 ... 113

Figure 4-3.2 Estimated Engel curve modelled by the PLTRM ... 114

Figure 4-3.3 Preston curve ... 117

Figure 4-3.4 Preston curve modelled by non-parametric models ... 118

(10)

Figure 4-4.1 Weight–height ratio against age with fitted PLTRM ... 124

Figure 4-4.2 Scatter plot of LIDAR data ... 126

Figure 4-4.3 LIDAR data fitted with different models: PLTRM; ... 127

Figure 4-4.4 The confidence intervals for α and sampling distribution of ˆα ... 129

Figure B.1 The regression splines basis for knot * * * * 1 0.2, 2 0.4, 3 0.6, 4 0.8 x = x = x = x = ... 144

Figure B.2 The derivatives of the regression splines basis for knots * * * * 1 0.2, 2 0.4, 3 0.6, 4 0.8 x = x = x = x = . ... 144 Figure C.1 PLTRM for DGP1:α= −0.5,β₀=2,β₁=1.5,n=33 ... 145 Figure C.2 PLTRM for DGP2: α= ,0 β₀=2,β₁=1.5,n=33 ... 146 Figure C.3 PLTRM for DGP3: α=0.475,β₀=2,β₁=1.5,n=33 ... 147 Figure C.4 PLTRM for DGP4: α= −0.5,β₀=2,β₁=3,n=33 ... 148 Figure C.5 PLTRM for DGP5*: α= ,0 β₀=2,β₁=3,n=33 ... 149 Figure C.6 PLTRM for DGP6: α=0.475,β₀=2,β₁=3,n=33 ... 150 Figure C.7 PLTRM for DGP7:α= −0.5,β₀=2,β₁=1.5,n=65 ... 151 Figure C.8 PLTRM for DGP8: α= ,0 β₀=2,β₁=1.5,n=65 ... 152 Figure C.9 PLTRM for DGP9: α=0.475,β₀=2,β₁=1.5,n=65 ... 153 Figure C.10 PLTRM for DGP10: α= −0.5,β₀=2,β₁=3,n=65 ... 154 Figure C.11 PLTRM for DGP11*: α= ,0 β₀=2,β₁=3,n=65 ... 155 Figure C.12 PLTRM for DGP12: α=0.475,β₀=2,β₁=3,n=65 ... 156 Figure C.13 PLTRM for DGP13:α= −0.5,β₀=2,β₁=1.5,n=129 ... 157 Figure C.14 PLTRM for DGP14: α= ,0 β₀=2,β₁=1.5,n=129 ... 158 Figure C.15 PLTRM for DGP15: α=0.475,β₀=2,β₁=1.5,n=129 ... 159 Figure C.16 PLTRM for DGP16: α= −0.5,β₀=2,β₁=3,n=129 ... 160 Figure C.17 PLTRM for DGP17*: α= ,0 β₀=2,β₁=3,n=129... 161 Figure C.18 PLTRM for DGP18: α=0.475,β₀=2,β₁=3,n=129 ... 162 Figure C.19 PLTRM for DGP19:α= −0.5,β₀=2,β₁=1.5,n=257 ... 163 Figure C.20 PLTRM for DGP20: α= ,0 β₀=2,β₁=1.5,n=257 ... 164 Figure C.21 PLTRM for DGP21: α=0.475,β₀=2,β₁=1.5,n=257 ... 165 Figure C.22 PLTRM for DGP22: α= −0.5,β₀=2,β₁=3,n=257 ... 166 Figure C.23 PLTRM for DGP23*: α= ,0 β₀=2,β₁=3,n=257 ... 167 Figure C.24 PLTRM for DGP24: α=0.475,β₀=2,β₁=3,n=257 ... 168 Figure C.25 PLTRM for DGP25:α= −0.5,β₀=2,β₁=1.5,n=513 ... 169 Figure C.26 PLTRM for DGP26: α= ,0 β₀=2,β₁=1.5,n=513 ... 170 Figure C.27 PLTRM for DGP27: α=0.475,β₀=2,β₁=1.5,n=513 ... 171 Figure C.28 PLTRM for DGP28: α= −0.5,β₀=2,β₁=3,n=513 ... 172 Figure C.29 PLTRM for DGP29*: α= ,0 β₀=2,β₁=3,n=513... 173 Figure C.30 PLTRM for DGP30: α=0.475,β₀=2,β₁=3,n=513 ... 174

(11)

Figure C.31 PLTRM for DGP31: α= −0.5,β₀=2,β₁=1.5,n=1025 ... 175 Figure C.32 PLTRM for DGP32: α= ,0 β₀=2,β₁=1.5,n=1025 ... 176 Figure C.33 PLTRM for DGP33: α=0.475,β₀=2,β₁=1.5,n=1025 ... 177 Figure C.34 PLTRM for DGP34: α= −0.5,β₀=2,β₁=3,n=1025 ... 178 Figure C.35 PLTRM for DGP35*: α= ,0 β₀=2,β₁=3,n=1025 ... 179 Figure C.36 PLTRM for DGP36: α=0.475,β₀=2,β₁=3,n=1025 ... 180

(12)

Acknowledgments

I sincerely want to express my utmost gratitude to my supervisor, Dr. David Giles, for his continuous support of my Ph.D. study and research, and for his patience, motivation, and excellent teaching. From his lectures and under his guidance, I learned a considerable amount and was inspired to learn more. I cannot help but believe that he must be the best professor of econometrics and supervisor in the world. I could not have imagined my Ph.D. study without him.

My sincere thanks also goes to Dr. Judith Clarke for her valuable comments during presentations of my research work at the department Brown Bag seminars as well as the valuable econometrics education I received from her.

Lastly, I would like to thank my family—my parents, brother, sister-in-law, and my wife—for their emotional support throughout my Ph.D. study.

(13)

Dedication

(14)

Global regression modelling is a general practice in the empirical literature. In global

regression modelling, a single parametric, non-parametric, or possibly semiparametric model is assumed to be appropriate over the whole domain of a given data set. However, this may not always be true. The way in which the explained variable depends on the explanatory variables (i.e., the relationship between the dependent variable and independent variables) may change over the sample. Such changes may depend on the values taken by some of the explanatory variable(s), or other variables. This suggests the use of a threshold regression model (TRM).

The existing threshold regression models, called traditional threshold regression models hereafter, allow multiple global models to function as global models in their local domains. We term this strategy of localizing “global” behaviour “non-global regression modelling.” However, in traditional threshold regression models, all sub-models must be parameterized. This requirement makes it risky to use the traditional threshold regression models under some circumstances. For example, the local behaviour is quite unique and is difficult to model via a parametric model. As long as there are any sub-models being wrongly specified in the

traditional threshold regression models, the estimators of all of the parameters will be inconsistent, and all inference undertaken will be incorrect and misleading. Because of the

(15)

fundamental flaws of the traditional regression models, we propose a new non-global regression model, namely the partial linear threshold regression model (PLTRM). The focus of the following three Chapters centers on the PLTRM.

In Chapter 2, the definitions of various “non-global models” are discussed, and certain selected non-global models and their limitations are reviewed. A general model setting for the PLTRM is provided. The PLTRM is divided into three groups: the jump partial linear threshold regression model (JPLTRM), the ordinary partial linear threshold regression model (OPLTRM), and the smooth transition partial linear threshold regression model (STPLTRM). To restrict this dissertation to a manageable extent, we limit our discussion to the OPLTRM; and a motivating example is provided in Chapter 2. We presume that we can find appropriate estimators (estimators that produce estimates that deviate from the true values of the

parameters to only a moderate extent) for the PLTRM and conduct Monte Carlo experiments. The simulation results strongly support the PLTRM for the simulated data sets that exhibit partial linear and partial nonlinear (with respect to the domain) characteristics.

The usefulness of a given model cannot be established until an appropriate and effective estimation method for the model is also proposed. With the general model setup presented in Chapter 2, we discuss the estimation method and some inference issues for the PLTRM in Chapter 3. An unbalanced penalized least squares method is found to be appropriate for the

(16)

PLTRM. The usual least squares method for global linear regression models and the penalized regression smoothing spline technique are combined judiciously to obtain this estimation method for the PLTRM. The estimation methods are discussed, respectively, based on the types of join, which are distinguished by the joining features. Due to the overall nonlinearity of the model, there is no analytical solution for the estimator of the threshold or for the estimator for the non-parametric segment. However, numerical estimates obtained from the designed estimation procedures are shown to be weakly consistent based on the parametric bootstrap results as long as there is no identification problem. Bootstrap sampling distributions of the estimators of the threshold and the parameters in the linear segment are provided in this Chapter and in Appendix C. In most of the cases, the sampling distributions of the estimators for the threshold are bell-shaped but more leptokurtic than a normal distribution. Moreover, the sampling distributions of the estimators of the parameters in the linear segment are very close to a normal distribution for most simulated data sets. Various inference issues such as hypothesis testing and confidence interval construction are also discussed along with the sampling distributions in this Chapter.

The contribution and the potential of a model can be only established in the real world; thus, in Chapter 4, we apply the PLTRM to several data sets from the real world. Among the data sets, we have both cross-sectional and time series data sets, and we have data sets from economics as well as from other areas. With the selected data sets, the PLTRM is shown to

(17)

be a powerful tool to model certain trends in time series data, to describe certain relationships between variables and to detect any change in functional forms.

The rest of this dissertation proceeds as follows. In Chapter 2, we motivate and propose the partial linear threshold regression model. In Chapter 3, we estimate the model and discuss the statistical properties of the resulting estimators as well as associated parameter inference issues. In Chapter 4, we apply the PLTRM to data sets from the real world and show how the PLTRM can be used for various purposes. In Chapter 5, we provide some general

(18)

Chapter 2 A New Type of Non-Global Regression Modelling: The

Partial Linear Threshold Regression Model

1. Introduction

In econometrics, most researchers conduct regression analyses based on a single global model that is assumed to be the true, or the best, model. Of course, a searching procedure among candidate models is usually involved beforehand. As a starting point, comparing the performances of different global models is a good idea, since it can provide us with basic information about the population. In addition, in some simple cases, the relative performance of the candidate models does not depend on explanatory variables or other variables (e.g., threshold variables). Furthermore, a global model that is believed to describe the data well usually has an easier interpretation and a better predictive power than that of non-global models because it does not involve uncertainty that is related to the change between

submodels. These are all good reasons for the practice of using global models (e.g., see Yang, 2008). However, no matter how thorough and deliberate the searching procedure is, this general routine practice may still be risky. The default assumption underlying this prevalent practice is that a single data-generating process or true model is sufficient to represent the features revealed by the population of interest. This is a fairly strong assumption in reality, and it is likely to fail to hold when facing high dimensional or complex data, in which complicated relationships among variables are more likely. The possibility of the failure of the global representation assumption is the reason behind the idea of non-global regression modelling.

(19)

Although the term “non-global” can be found occasionally in the econometrics literature, it is not yet well defined. To avoid confusion, it is necessary to clearly define this term with which this and subsequent Chapters will be concerned. Like the term “linear” in the context of “linear regression,” “non-global” is a vague concept if one does not specify “in terms of what.” Broadly speaking, a non-global regression model can be thought of as any regression model that does not have consistent characteristics in one or more aspects. According to this definition, the linear regression with heteroscedasticity, the partial linear model, and the threshold regression model can all be viewed as non-global regression models, since they are non-global in terms of scedasticity, parameterization, and domains (and linearity),

respectively. However, we use the term “non-global” in a narrow sense to refer in particular to the models that are non-global in terms of domains, but may or may not be non-global in terms of other aspects. In other words, a necessary condition to be qualified as our defined non-global model is non-global domains. Hereafter, the term “non-global” is used in this narrow sense and the term “global” is used as its opposite.

Non-global regression models, by relaxing the global representation assumption and allowing for different functional forms within different “sub-populations” or “regimes” that are

classified by the domains of the functions, may improve the flexibility of econometric modelling. Based on whether the boundaries of the domains are known or not, non-global models can be further divided into two groups. However, the group with known boundaries has attracted less interest, because, in this case, the models are just simple combinations of conventional global parametric models. The typical cases being widely studied are those in which the boundaries are unknown and must be estimated.

(20)

The earliest non-global regression can be seen in the structural-break or change-point literature. The structural-break problem was first studied in quality control by Page (1954, 1955). In econometrics, this usually arises in the context of time series data. Failure to detect possible structural-breaks may result in huge forecasting errors and model misspecification. The well-known Chow test proposed by Chow (1960) shows the evidence of early non-global modelling. At the early stage of study, traditionally, structural-break problems were usually converted into hypothesis testing problems. Structural stability is assumed under the null hypothesis, and one or more structural breaks are assumed under the alternative. Once the null hypothesis is rejected, the location(s) of the break(s) must be estimated. The literature that focuses on this traditional practice include Andrews (1993, 2003), Bai and Perron (1998), and Hansen (2002). Lately, structural-break estimation has also been viewed differently as a model selection problem (e.g., see Davis et al., 2006; Lu et al., 2010). For a general review of the recent work on structural breaks in time series, one may refer to Aue and Horváth (2013).

The threshold regression model (TRM) is naturally non-global, since a prominent property of the threshold regression model is its non-global domains. In fact, the structural-break

problems can also be considered to be threshold regressions to some extent, since almost all structural-break problems can be converted into threshold regression models. In contrast to the structural-break models, which usually do not make sufficient specific assumptions about the transitions between regimes, the threshold regression models usually explicitly posit the transitions (Geweke and Jiang, 2011). In the literature, there is a tendency to study similar problems in both the fields of structural breaks and threshold regressions. Although the

(21)

threshold regression model can be dated back to Quandt (1958) and Dagenais (1969), it did not draw much attention until a formal introduction by Tong and Lim (1980). For this reason, Tong and Lim are usually considered to be the first to introduce the threshold regression model. The Self-Exciting threshold autoregressive (SETAR) model (Tong and Lim, 1980), the smoothing transition autoregressive (STAR) model (Teräsvirta, 1994), and the Markov switching (MS) regression model (Goldfeld and Quandt, 1973) are three of the most popular models used in econometrics to model nonlinearity. The threshold regression models have occasionally been used to model non-global error structures as well. One example is the double-threshold ARCH model (DTARCH), which can be used to handle a situation where both the conditional mean and the conditional variance are piecewise linear specifications (see e.g., Baragona and Cucina, 2008; Li and Li, 1996; Wong and Li, 2000). For a brief review of the threshold regression model in time series, one can refer to Tong (2011). In addition, there are a few papers discussing the threshold models in more general settings rather than in time series (e.g., Caner and Hansen, 2004; Hansen, 2000, 2002; Seo and Linton, 2006; Kourtellos et al., 2011; Yu, 2012).

Non-global regression models, thus far, can be categorized as structural-break models or threshold regression models in general. Whether a model is a structural-break model or a threshold regression model, the parameterization of the sub-models within regimes is always assumed. Undoubtedly, this assumption may incur some risks associated with model

misspecification in many circumstances, as we may not have complete knowledge to correctly specify the functional forms for the regimes, especially for cases in which certain sub-models have quite nonlinear functional forms. We need to bear in mind that the purpose

(22)

of using a non-global regression model is to reduce the chance of model misspecification resulting from a global specification. However, the pure parametric assumption of the traditional non-global regression models may increase the probability of model misspecification in the sub-models. If the sub-models are severely mis-specified, the advantage of using a non-global regression model over a global regression model may be completely ruined or even lead to a worse situation. For the sake of retaining as many advantages of the non-global regression models as possible, it calls for a new type of non-global regression models, in which nonparametric models might be allowed for the regimes.

The partial linear threshold regression models (PLTRM), by introducing the semiparametric idea into the traditional threshold regression models, allow for both linear parameterized segments and non-parametric segments. The use of both parametric and nonparametric functions to model the regimes in a threshold regression model is a significant difference between the PLTRM and other closely related models, in the sense of combining of semiparametric ideas and threshold regression tools. Examples include those models discussed in Campbell (2002), Xia et al. (2007), Eichler et al. (2013), and Cai et al. (2015). The models in Campbell (2002) and Eichler et al. (2013) are similar. They both model the distribution of the independent variable(s) conditional on the regime nonparametrically while keeping parametric modelling for the regimes in the Markov switching model context. Xia et

al. (2007) proposed a nonparametric method to estimate the threshold variables by allowing nonparametric functions to model the varying coefficients in a linear regression model. Cai et

(23)

semiparametrically, to forecast exchange rates. However, in the related literature, there are no models allowing for both parametric and nonparametric functions to model regimes in a threshold regression model as that in the PLTRM. As a new class of non-global regression models, the PLTRMs may effectively solve the problems arising in traditional non-global regression models without a loss of efficiency, which may be encouraging.

This Chapter is organized as follows. Section 2 describes the possible limitations of the typical global regression models. Section 3 shows how traditional non-global regression models model the change in regimes, and discusses the advantages and the disadvantages of using traditional non-global regression models. Section 4 describes the partial linear

threshold regression model and its setup. Section 5 motivates the PLTRM by providing an illustration and some performance evaluations. Section 6 concludes.

2. The Limitations of Global Regression Models

Global regression models have their own advantages and disadvantages in the global domain context. They also have some common limitations in application in more general settings, in which non-global domains are allowed.

Regular parametric models and non-parametric models are the two most important and widely used regression models in econometrics, and they have their own pros and cons. The former assumes that the conditional mean of the dependent variable can be represented by a purely parameterized function, which could be linear or nonlinear in terms of the parameters.

(24)

The latter, on the other hand, discards the assumptions about the functional form. Under stronger assumptions, stronger conclusions can also be drawn from the global parametric model. However, the conclusions could be completely inaccurate and misleading if the model is misspecified. On the contrary, only weak conclusions can be made, due to the weaker assumptions being assumed in the global non-parametric model. It is obvious that these two types of models are appropriate only for the two extreme situations, respectively, under the global domain. The global parametric model works best when the modeller has complete knowledge about the functional form according to some economic theories or past research, while a global non-parametric model works best when the modeller has no information about the functional form.

In practice, the actual situation is usually in between these two extremes. Modellers usually have limited, not complete, knowledge about the functional forms. In this case, a

semiparametric model that assumes the functional forms are partly known may suffice. This model forms the third type of global regression model (i.e., the semiparametric model). The partial linear model introduced by Engle et al. (1986) is a good example of a semiparametric model. In a partial linear model, the functional form with respect to a variable of particular interest is problematic and thus remains unspecified, while the functional form with respect to other variables is assumed to be linear and is thus specified as a linear function with a finite number of unknown parameters. For a broad review of semiparametric models, one can refer to Yatchew (2003). It is worth mentioning that the semiparametric regression model is a non-global regression model according to the broader definition but is not according to the narrow one.

(25)

The global regression models described above have their own strengths and weaknesses in dealing with various situations. In particular, due to their global domain property, they have two common weaknesses: (1) sub-optimal performances measured in terms of mean square error (MSE) when the functional forms depend on the locations of the data and (2) an inability to estimate the thresholds driving the change of the functional forms. To illustrate the limitations of global regression models, let us look at two examples using real data.

Example 2-2.1 The data plot in Figure 2-2.1 consists of 72 measurements on the age in months and the weight-height ratio of pre-school boys (Gallant, 1977). The plot shows a J-shaped relationship between the weight-height ratio and the age. From the plot, we can see that the weight-height ratio behaves differently as the age exceeds certain months. When the children are less than approximately 12 months, the data suggests a quadratic response of the weight-height ratio to the age. However, when the age exceeds 12 months or so, the data suggests a linear relationship.

The plot suggests that a single global parametric, non-parametric, or a semiparametric model may not be an optimal choice. On one hand, it seems there is no single, simple parametric function that we can come up with to represent an exact J-shaped curve. On the other hand, it is difficult to incorporate the partial linear information contained in the J-shaped function into a global non-parametric or a semiparametric model. The non-parametric and semiparametric representation of the linear regime tends to under smooth the data for the linear segment.

(26)

0 10 20 30 40 50 60 70 0 .5 0 .6 0 .7 0 .8 0 .9 1 .0 Age (months) W e ig h t-H e ig h t R a ti o ( p o u n d s /i n c h )

Figure 2-2.1 Weight-height ratio versus age for preschool boys:

ⅰ ⅰⅰ

ⅰLinear fit (green); ⅱⅱⅱCubic smoothing spline (red) ⅱ

As any useful information has its intrinsic value in econometrics, ignoring or failing to utilize such information will likely lead to inefficiency in estimation. From the linear fit and the cubic smoothing spline in Figure 2-2.1, we can see that the linear model is a better

representation of the relationship between the age and the weight-height ratio when the age exceeds 10 months. The cubic smoothing spline captures the characteristics of the data better for the non-linear segments (i.e., age under 10 months). However, the overall fit to the data is notably inferior if only the linear model, or only the smoothing spline, is used for the full sample. The unbiasedness and consistency of the ordinary least squares (OLS) estimator are destroyed by the nonlinear segments, and the cubic smoothing spline is under smoothed for the linear segments because of the use of the global smoothing parameter. Therefore, traditional global models are inappropriate for data sets of this type.

(27)

In addition, we may also be interested in the location at which the response of the weight-height ratio to the age is changed for some practical reasons (e.g., good nutrition suggestions for pre-school boys to achieve a better body shape based on the two age periods distinguished by the growing patterns of the weight-height ratio). However, global regression models are usually incapable of detecting this kind of change.

Based on the information we have from the data plot, a non-global model that allows different parametric functional forms, depending on the locations of the data, seems more appropriate in this case. Gallant (1977) fitted the data with both a quadratic-linear model and a quadratic-quadratic linear model. Both of them perform well in terms of mean square error (MSE) in this specific situation. Model selection among the competing non-global regression models with different regimes is beyond the scope of this dissertation.

Example 2-2.2 Figure 2-2.2 from Silverman (1985) plots the head acceleration in

g(g-force)1 versus the time in milliseconds since impact, which are collected from a simulated motorcycle accident. From the data, we can see that a global linear regression model and a parametric nonlinear model are strikingly inappropriate due to the dramatic nonlinear characteristics of the data.

In fact, this example is commonly used to illustrate the usefulness of the smoothing spline, which is a global non-parametric model. The fitted cubic spline seems to describe the data quite well. However, from the graph we can also see that when the time is less than about 11

1

(28)

milliseconds, the plot suggests linear characteristics. If we can incorporate this information into the model, we may improve the performance of the model in terms of fit.

10 20 30 40 50 -1 0 0 -5 0 0 5 0 Time (ms) A c c e le ra ti o n ( g )

Figure 2-2.2 Motorcycle-helmet impact data with a cubic smoothing spline

We chose a few values of time around 10, and each time used one as a splitting point to split the data into two subsets; we then fitted the first subset with a linear model and the second with a cubic smoothing spline. We found that the sum of squared residuals is a little bit smaller than that obtained from the global non-parametric model when 11 is selected as the splitting value. The experiment shows that potential gains may be achieved by using non-global modelling.

(29)

3. Non-global Regression Modelling

3.1 A General Form for Non-global Regression Models

Non-global regression models are used to study the stochastic relationship between a dependent variable and one or more independent variables when the relationship also depends on the locations of the data. The generic form of the non-global model is

1 1 1 1 2 2 2 2 ( ; ), Z R ( ; ), Z R ( | , ) ( ; ; R(Z, )) ( ; ) Z R . M M M M f X f X E y X Z f X f X θ θ θ α θ ∈   ∈  = =    _∈  ⋮ ⋮ (2-3.1)

Here, y is the dependent variable and

1 M m m X X =

=

_∪

is a set of covariates. The function

( ; ; R(Z, ))

f X θ α is defined by M different continuous and smooth functions

( ; )

m m m

f X θ ,m= ⋯1, ,M over different domains D ,_m m= ⋯1, ,M. Each domain is uniquely determined by the threshold region R_m, m= ⋯1, ,M. The threshold variable(s) Z and the threshold vector

α

together define theR_m,m= ⋯1 M. Both D_m and R_mare closed and

well-defined sets such that

1 M m m D D = =

∪

,Dm∩Dn = ∅ ∀, m≠n and 1 M m m R R = =

∪

, , m n

R ∩R = ∅ ∀m≠n. The variablesXand Z may or may not have elements in common.

The general form of the non-global regression model (2-3.1) seems too general and

complicated to proceed. To better understand the idea of non-global regression modelling, let us consider a special case, where X₁=X₂ =⋯=X_M =X =Z =

{ }

x contains a single

(30)

variable x ; and Dm =Rm ={ |x αm−₁< ≤x αm} with α0 =min( )x and αM =max( )x . In

this case, we can write a simplified version of (2-3.1) (Seber and Wild, 1989, p.433) as

1 1 1 2 2 1 2 1 ( ; ), ( ; ), ( | ) ( ; , ) ( ; ) . M M M f x x f x x E y x f x f x x θ α θ α α θ α θ α − ≤   < ≤  = =    _<  ⋮ ⋮ (2-3.2)

Under “hard thresholding”, (3.2) can also be expressed in an equivalent short form as

1 1 ( | ) ( ; , ) ( ; ) ( ) M m m m m m m E y x f xθ α f xθ I α ₋ x α = = =

∑

< ≤ , (2-3.3)

where Im( )A is an indicator function that takes a value of 1 if A occurs and 0 otherwise. It acts as the switching mechanism in non-global modelling.

With this simplified version of a non-global regression model, we can discuss most modelling issues more easily in a relatively simple context.

3.2 Some Non-global Regression Models

Based on whether or not the thresholds are known, non-global regression models can be divided into two groups. However, most modelling issues can be treated as in the global context if the thresholds are known. For example, we can use dummy variables to deal with the non-global issues (i.e., different parameters for different groups) when the threshold variables are discrete and categorical rather than continuous. We can use the Chow test to approach the structural-break problem with a single known break in the mean of the dependent variable for stationary processes with the same variance in both regressions. In

(31)

this Chapter, we will focus on the second case, in which the thresholds are unknown and need to be estimated.

M M M

M _{1 A Structural-break model with unknown break(s)}

A structural-break model (sometimes called a change-point model) is a model that is

commonly used to address the problems of unstable parameters in a time series model. In the literature, it is sometimes inappropriately referred to as a multi-phase model, a switching model, or a segmented regression model due to a deficient specification (Khodadadi, 2008).

A general structural-break model with m breaks (m+ regimes) can be expressed in the 1 form of (2-3.2) as ' ' 1 1 1 ' ' 2 1 2 1 2 ' ' 1 , ( 1, , ) , ( 1, , ) ( | , ) ( 1, , ). t t t t t t t t t m m m x z t T or t T x z T t T or t T T E y x z x z T t or t T T β δ β δ β δ +  ₊ _≤ ₌  + < ≤ = +  =    ₊ _< ₌ ₊  ⋯ ⋯ ⋮ ⋮ ⋯ (2-3.4)

In this model, y is the dependent variable. _t x pt( ×1) and z qt( ×1) are vectors of

covariates. β is the vector of coefficients of x , which is assumed to be stable over the _t

whole sample. δj(j=1,⋯,m+1) are vectors of coefficients of z , which are subject to t

change across sub-samples that are determined by the breaks Tj (j= ⋯1, , )m . This partial structural change model can be modified to a pure structural-break model by setting p= . 0 A striking feature of this model is that the structural-break points (T ) as well as the vectors _j

(32)

Structural-break problems have drawn much attention in econometrics. Among the related articles, most of them are designed for the case of a single break point. For an extensive review in this area, one can refer to Perron and Qu (2006). Multiple structural-breaks have also received some attention in recent years. Bai and Perron (1998) discussed how to estimate and test for multiple structural changes in 1998 and extended their work in later years (e.g., Bai and Perron, 2003a; Bai and Perron, 2003b; Perron and Qu, 2006; and Perron, 2007). In addition, Andreou and Ghysels (2009), Aue and Horváth (2013), Ciuperca (2011), Furno (2012), Geweke and Jiang (2011), and Oka and Qu (2011) have also explored this field in recent years.

M M M

M _{2 The threshold regression models}

In contrast to the structural-break model, in which time is the only factor driving the change of regimes, in the threshold regression model all variables, including the lagged values of the dependent variable, can be used as threshold variables to drive the change of the functional forms across “regimes” or “classes.” In fact, the structural-break model can be considered to be a special case of the threshold regression model. For the sake of common usage, and a separate name being used in the literature, researchers usually treat it as a separate model from the threshold model.

Piecewise linear threshold regression models or segmented linear regression models are most commonly used models in non-global modelling. For a simple case with only one regressor and one threshold variable, we can write the model as

(33)

10 11 20 21 , ( | ) , . i i i i i x z E y x x z β β α β β α + ≤  =  + >  (2-3.5)

The variables and parameters in (2-3.5) are defined similarly to those in (2-3.2), except that we have a different threshold variable z here. A model like (2-3.5) is usually called a _i

structural threshold regression (STR) model. Different assumptions (i.e., exogenous and endogenous) about the regressors and the threshold variable will lead to different

specifications. One can find an example of the STR with an exogenous threshold variable in Hansen (2000). To learn more about STR with endogeneity in both regressors and the threshold variable, one may refer to Kourtellos et al. (2011).

With some minor change for (2-3.5), we can obtain another model

10 11 1 20 21 1 , , . t t t d t t t t d X X X X X β β ε α β β φε α − − − − + + ≤  = + + >  (2-3.6)

Equation (2-3.6) is a well known model in time series modelling and is called the Self-Exciting Threshold Autoregression (SETAR) model. The SETAR model was first proposed by Tong and Lim (1980) and has been used widely in time series modelling. A lot of models such as STAR, TARMAX, NeTAR, SETARMA, TVSTAR, and DTARCH have been developed and have joined the family of threshold regression models. A brief review of threshold models in time series modelling can be found in Tong (2011).

A regime switching model (Quandt and Ramsey, 1978) can be considered to be an extension of the STR model, in which the switching mechanism is replaced by a probability. A simple switching model with two regimes can be written as

(34)

10 11 20 21 , ( | ) , 1 ; i i i x with probability E y x x with probability β β λ β β λ +  =  + −  , (2-3.7)

where 0≤λ≤ . If 1 λ= or 0 λ=1,the model reduces to a standard global model. In this model, the functional forms depend on certain probabilities rather than the threshold variable. However, it cannot take both functional forms at the same time. A regime switching model will not be considered to be a non-global model that we defined earlier, as it is global in terms of domains.

Most threshold models we have discussed thus far are piecewise linear. However, nothing prohibits us from taking nonlinear functional forms for the sub-models. In practice, we may have some regimes taking linear functional forms and others taking nonlinear forms.

3.3 Modelling the Changes in Non-global Regression Models

An important and attractive feature of non-global regression models is that different

functional forms within different domains are allowed. This feature enriches the flexibility of the functional forms. Meanwhile, how to combine different functional forms together over the collectively exhaustive and disjointed domains becomes an unavoidable issue. In this sub-section, we discuss different ways to model the changes between the regimes in non-global regression models.

The changes between regimes can be either noncontinuous or continuous. Noncontinuous changes mean that the functions for any two adjacent regimes have different functional values at the threshold. In fact, non-global models expressed as (2-3.2) and (2-3.3) are

(35)

generally only piecewise continuous but not at the boundaries of the regimes if no further continuity constraints are being imposed. In other words, the functions fm(Xm;θm),

1, ,

m= ⋯ M are all continuous but the conditional expectations E y x are not. Figure ( | ) 2-3.1 shows an example of a possible noncontinuous change of a non-global regression model with two regimes. An abrupt change seems to appear in the relationship between the dependent variable y and the independent variable x , with this discontinuous, abrupt change seeming to lead a jump between the linear and the nonlinear regimes.

-1.0 -0.5 0.0 0.5 1.0 -1 .0 -0 .5 0 .0 0 .5 1 .0 1 .5 2 .0 x y

Figure 2-3.1 Non-global model with discontinuous change

There are three possible ways to model the continuous changes of non-global models: abrupt, smooth, and smooth transition. The first two types of changes can be achieved by imposing restrictions on functional values and first derivatives at the threshold. Let us use model (2-3.2) to illustrate how we can force the functions for any two adjacent segments to join together abruptly or smoothly. Since each function is continuous within its domain, the global

(36)

continuity of E y x will be guaranteed if we ensure the continuity at the thresholds. Thus, ( | ) we can just impose the constraints

fm(α θm; m)= fm+₁(α θm; m+₁) , m=1,⋯,M−1. (2-3.8) Equation (2-3.8) implies that the function values evaluating at the boundaries for any two adjacent regimes are equal. This avoids a situation like that shown in Figure 2-3.1. Under the constraints of (2-3.8), a two regime non-global model is shown in panel (a) of Figure 2-3.2, with the graph no longer showing any jump. However, the joint seems odd in the sense that the change between the regimes is not smooth. To achieve the smoothness of the change, additional constraints of the form

( ; ) 1( ; 1) m m m m m m x x df x df x dx _α dx _α θ + θ + = =   =     ,m=1,⋯,M−1 (2-3.9) must be imposed. The equations in (2-3.9) are usually called “smoothing” constraints, which means the non-global model will behave nicely and smoothly under this condition and (2-3.8). In this case, not only is the function E y x continuous, but so too are the first ( | ) derivatives. A model with these properties can be found in panel (b) of Figure 2-3.2.

-1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 x y (a) (b) Figure 2-3.2 Non-global model with continuous change:

(37)

One important model worth mentioning here is that based on the segmented polynomials. In this model, individual sub-models are polynomials of the regressors. We can easily

incorporate the continuity constraints (2-3.8) and the smoothing constraints (2-3.9) into the model, by considering a model in the following form:

,

( ) j ( )r

j m r m

j m r

f x =

∑

φ x +

∑∑

φ x−α ₊, (2-3.10) with r≥2 and where (x−α_m)r+=(x−α_m)r×I x( >α_m). A q-spline is a special segmented polynomial model in which every polynomial sub-model has degree q and is q− times 1 differentiable.

An alternative way to model smooth and continuous changes between regimes is through the use of a smooth transition function, such as the logistic transition function, the exponential transition function, the sign function, the hyperbola and hyperbolic functions. Here, we use a hyperbolic tangent function, tanh( )⋅ , as our smooth transition function for illustrative

purposes. A hyperbolic tangent transition function, is tanh(x a)

b −

,

with tanh( ) z z z z e e z e e − − − = + .

(2-3.11) Here, a is the location parameter that determines the switch point for the two sub-models, and b is the smoothing parameter that controls the level of smoothness. Figure 2-3.3 shows the curves for tanh(x a)

b

−

with a= and different levels of smoothness (Seber and Wild, 0

(38)

Figure 2-3.3 Hyperbolic tangent transition tanh(x a)

b

−

for a=0 and b=1, 2, 3

From Figure 2-3.3, we can see that the function exhibits less curvature as we increase ,b

and it is always restricted to lie between -1 and 1. With any smooth transition function,s(e.g., tanh() function), a smooth transition model with two regimes f and ₁ f can be expressed ₂

as

f =sf₂+ −(1 s f) ₁. (2-3.12) Figure 2-3.4 shows an example in which f₁ is a linear function (e.g., f x₁( )= + ) and 5 x f₂

is a nonlinear function (e.g., f x₂( )=sin(2 )x ); and the non-global model behaves as f₁ for

1

x≤ and f for ₂ x> with a smooth transition function 1 s x( ) 0.5 0.5 tanh(x 1)

b

−

= + . Two

(39)

bigger b , s x as well as ( )( ) f x exhibit a less abrupt change. In panel (b), as b approaches 0,

the model is close to a non-global model with an abrupt change, as shown in Figure 2-3.4.

(a) (b) Figure 2-3.4 Non-global smooth transition model with s x( ) 0.5 0.5 tanh(x 1)

b

−

= +

3.4 Advantages and Disadvantages of Non-global Regression Modelling

Compared with the traditional global model, a non-global model relaxes the assumption of a single global representation and combines different global models within sub-domains in a flexible way. This may allow researchers to deal with more complex situations in the real world. In addition, non-global models encompass global parametric models (i.e., linear and nonlinear models) as special cases. When the threshold value is estimated to be out of the range that the threshold variable takes in the sample, a non-global model reduces to a global parametric model. Moreover, a non-global regression model provides us with the opportunity to investigate the change between regimes, which may be quite important in certain

(40)

example in which people have a particular interest in the regime change. Smith and Cook (1980) discussed how to use a non-global model with the reciprocal of serum-creatinine and time after a renal transplant as the dependent and independent variables, respectively, to detect the time of the rejection of the kidney. Finally, it is usually less risky to start with a non-global regression model instead of a global model if we do not know the true model is global or non-global. We may lose some efficiency of the estimators if non-global regression model is used while the true model is actually global. However, the estimators we obtain are usually biased and inconsistent if we fit with a global model when in fact the true model is non-global. In addition, a general-to-specific test procedure may help us to test a non-global regression model down to a global model if the true model is global. We will discuss this point in detail when we are describing the partial linear threshold model in the next section.

An obvious disadvantage of non-global regression is that the modelling may be complicated, because more parameters and more issues need to be considered. In addition, the computation cost is usually high in a non-global model, since usually more parameters need to be

estimated and more complex estimating procedures are involved. With the development of non-global regression modelling and increasing computing power, these two disadvantages may not be limiting. However, a major disadvantage can be found in the existing non-global modelling if we carefully investigate the models. Without exception, all existing non-global models assume a parametric functional form for each regime. On the one hand, this

assumption limits the flexibility of the functional form modelling. On the other hand, it increases the possibility of introducing a model misspecification problem, since the whole

(41)

model is completely correctly specified if and only if we correctly specify the model in each regime, which is usually unlikely in practice.

In low dimensional cases, basic data plots usually can help us to learn about possible functional forms. However, when we are facing high dimensional or complex data, we may have difficulty identifying possible regimes as well as potential functional forms for each regime. Imprudently specifying the functional forms is risky. Therefore, it may be more reasonable to leave the functional forms unspecified if we do not have well-specified information about the functional forms for some segments.

4. The Partial Linear Threshold Regression Model

As an extension to current non-global regression models, the partial linear threshold

regression model (PLTRM) combines semiparametric modelling and non-global modelling. It is the first to achieve the compatibility of parametric and non-parametric models in

non-global regression modelling. For simplicity, we will focus on the two-regime models, in which one segment takes the linear functional form while the other takes no parameterized functional form. Due to the different ways of modelling the change between the regimes, the PLTRM can also take different forms. This leads to a family of PLTRMs, including the jump partial linear threshold regression model (JPLTRM), the ordinary partial linear threshold regression model (OPLTRM), and the smooth transition partial linear threshold regression model (STPLTRM).

(42)

A JPLTRM with two regimes and single covariate can be written as

yi =(β₀+β₁x I xi) ( i <α)+m x I x( ) (i i≥α)+ui, (2-4.1) where y_i is the dependent variable, and x_i is the independent variable as well as the threshold variable. The variables u_i are the random disturbances that are independently and identically distributed with 0 mean and a finite variance. The parametersβ₀ and β₁ are the intercept and slope coefficients for the linear parametric segment, and αis the threshold, which is a fixed unknown constant that needs to be estimated. The functionm( )⋅ is a

continuous smooth function with unknown functional form. Equation (2-4.1) encompasses a global linear model and a global non-parametric model as special cases. Equation (2-4.1) collapses to a simple linear model when α >max( )xi and to a non-parametric model

whenα<min( ).xi In fact, all of the members of the family of PLTRMs have this desirable

property. Because of this property, the PLTRM can be used as a mechanism for model selection.

We can derive the ordinary partial linear threshold regression model (OPLTRM) by imposing a continuity restriction

m( )α =β₀+β α₁ . (2-4.2)

If we multiply both sides of (2-4.2) by I x( i ≥α) and add to (2-4.1), we can obtain another form for the OPLTRM:

yi =(β₀+β₁x I xi) ( i <α) [ ( )+ m xi +β₀+β α₁ −m( )] (α I xi ≥α)+ (2-4.3) ui

or

(43)

Equations (2-4.3) and (2-4.4) have a great advantage in conducting simulations. A data generating process (DGP) given by them naturally guarantees that the two sub-models are joined at the threshold, even though the two regimes are not both modelled by parametric functions.

Similarly, to get the smooth model, we can add the smoothness constraint

β₁=m'( )α (2-4.5) to (2-4.4). However, in this case a one line representation for the model may not be a good idea, since (2-4.5) involves evaluating the first derivative of the unknown function at α.

Given a smooth transition function ( )s x_i , a STPLTRM is expressed as

yi =(β₀+β₁xi)(1−s x( ))i +m x s x( ) ( )i i +ui, (2-4.6) where the parameters and variables are defined as above.

An obvious advantage of the PLTRM over traditional non-global models is that it removes the risk we may face when specifying the functional forms for nonlinear segments. In addition, it generalizes two widely used models: the linear model and the non-parametric model. Under the effective estimation procedure, a PLTRM model turns out to be a global model if the true DGP is a global model rather than a non-global model. For this reason, it is safer to start with a PLTRM when we are not quite sure about the true model. The estimation procedure itself is a model selection process, which involves selecting the most appropriate model from the competing models: the global linear model, the global non-parametric model,

(44)

and the non-global partial linear threshold regression model. The prominent advantages of the PLTRM make it a competitive model.

Within the family of the PLTRMs, estimation of the JPLTRM is less of interest, because the jump at threshold makes it easy to estimate. The STPLTRM, though might be of interest, is the most complicated one among the three models. As a start of the research of the PLTRMs, it seems not a good choice. The OPLTRM seems most appropriate to be explored first as a starting research of the PLTRMs because of the moderate complexity of the model and the possible applications to the real world data as suggested by the data sets used in Chapter 4. For these reasons, we focus on the OPLTRM in this and subsequent Chapters of this dissertation.

5. A Motivating Illustration

Consider estimating a regression model based on 129 observations. The x values are equally spaced on [ 1,1]− , and the true regression function or DGP is specified as

y_i =(2 3 ) (+ x I x_i _i <0) [sin(7 ) 2] (+ x_i + I x_i ≥0)+u_i, (2-5.1) where u_i ∼iid N(0, 0.5 )2 . A typical realization of the data and the true regression function are given in Figure 2-5.1.

(45)

-1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y

Figure 2-5.1 Scatter plot of a typical realization of 129 observations and the true model

-1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y Regression Fits True Linear SS PLT SL PL QD

Figure 2-5.2 Scatter plot and different fits: iiii true model-True (black); iiiiiiii linear model-Linear (blue); iii cubic smoothing spline-SS (red); iv partial linear model-PL (purple); v segmented linear

model with two break points-SL (brown); vi quadratic B-spline-QB (golden); vii partial linear threshold regression model-PLTRM (green)

(46)

The linear trend is obvious for the left part of the data but shows nonlinearities as well for the rest of the data. Naturally, one may fit the data with a simple linear regression model

ignoring the nonlinear characteristics or fit the data with a non-parametric or semiparametric method, assuming no functional form at all or partially known functional form, respectively. Experienced researchers may also choose a non-global model such as the segmented linear model to capture the partial linear trend revealed by the data plot. Alternatively, with the introduction of the new type of non-global regression, one may fit the data using a PLTRM to capture the location oriented characteristics of the data. Different model fits (e.g., linear model, cubic smoothing spline, partial linear model, segmented linear model, quadratic B-spline and the PLTRM) for the data set together with the true model are plotted in Figure 2-5.2.

To better compare with the true model, individual fits against the true model are also plotted in Figure 2-5.3. From Figure 2-5.2, it is shown that the PLTRM we proposed seems to fit best in terms of MSE for the given data set amongst all fitted models, with the true model as the reference. The linear model in this case has a poor fit, as is shown in panel (a) of Figure 2-5.3. It does not only over smoothes the nonlinear part of the data but also fails to correctly estimate the marginal effect of x on y. As a result of misspecification, OLS estimators are biased and inconsistent. The cubic smoothing spline, as shown in panel (b), fits the right half of the data quite well but tends to be under-smooth for the left half. The quadratic B-spline

(47)

-1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y True vs Linear True Linear -1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y

T rue vs Cubic smoothing spline

True 3SS

(a) (b) -1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y

T rue vs Partial linear threshold

True PLT -1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y

T rue vs Partial linear

True PL (c)

(d) -1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y

T rue vs Quadratic B-spline

True QB -1.0 -0.5 0.0 0.5 1.0 -1 0 1 2 3 x y

T rue vs Segmented linear

True SL

(e) (f) Figure 2-5.3 Fits against true model

has similar problems, as can be seen from panel (e). The segmented linear model shown in panel (f) seems to fit well, especially for the far-left part of the data.