Testing the shape of a regression curve

(1)

Testing the shape of a regression curve

Citation for published version (APA):

Diack, C. A. T. (2000). Testing the shape of a regression curve. (Report Eurandom; Vol. 2000031). Eurandom.

Document status and date: Published: 01/01/2000

Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne

Take down policy

If you believe that this document breaches copyright please contact us at:

openaccess@tue.nl

(2)

TESTING THE SHAPE OF A REGRESSION CURVE Cheikh A.T. Diack

EURANDOM, P.O. Box 513, 5600 MB Eindhoven, The Netherlands e-mail: diack@eurandom.tue.nl

||||||||||||||||||||||||||||||||||||||||||

Abstract

This paper proposes a hypothesis testing procedure for nonparametric regression models based on least squares splines. We assume that the sample is a part of stationary sequence which satisfy a mild mixing property. The approach yields tests of monotonicity and convexity.

Resume

Nous proposons une proc edure de test pour la fonction de regression bas ee sur les splines de r egression. Nous supposons que l' echantillon est une partie d'une suite stationnaire qui satisfait des conditions m elangeantes. Notre approche fournit un test de monotonie et de convexit e.

Key words: Testing monotonicity, Convexity, Central limit theorem, B-splines, Mixing

. ||||||||||||||||||||||||||||||||||||||||||

1 Introduction

In a variety of statistical models, a regression relationship can be assumed to be monotone or convex. A natural question is whether the available data support these assumptions. Therefore, testing monotonicity or convexity provides a way to prevent wrong conclusions. Some papers in statistics literature deal with nonparametric hypothesis tests for convexity or monotonicity of the regression function. Schlee (1982) proposes tests based on the greatest discrepancy between kernel type estimates of the derivatives of the response variable and zero. However, this paper lacks a discussion on consistency and conservativeness. Yatchew (1992) develops tests (with a semi-parametric model) based on comparing the nonparametric sum of squared residuals under monotonicity constraints, with the nonparametric sum of squared residuals without contraints. The Yatchew approach relies on sample splitting which results in a loss of eciency. Yatchew and Bos (1997) avoid this drawback, essentially by doing an unrestricted nonparametric regression using the residuals from the restricted regression, then testing for signicance. However, Yatchew and Bos' test does not have a good power asymptotically. Using a kernel type estimator, Bowman, Jones and Gijbels (1998) developed a test (of monotonicity) based on the size of a critical bandwidth. Bootstrapping is used to calculate the null distribution of the test statistics. The major drawback with this test is that its actual level is not guaranteed and its power can be low when there are at parts in the regression function. Moreover, asymptotic theory is not provided. Besides, all these tests assume that the random variables in their models are independent.

We consider the following regression model

Yi=g(xi) +Zii= 1 n:

The design pointsfx i

g n

i=1 can be deterministic or random. Without loss of generality, we assume

that xi

201]. We also assume thatfZ kk

2Zgis a strictly stationary sequence of real random

variables with zero mean on a probability space (AP). Let

k = EZ

iZi+k

be its covariance sequence. Let(Zii < 0) and(Zii

j) be the-elds generated byfZ ii <0

g

andfZ ii

jg respectively. We assume that the sequencefZ kk

2Zgis-mixing, that is:

j= sup A2(Z i i<0) B2(Z i ij) jP(AB);P(A)P(B)j!0 asj!+1:

We also assume that their spectral density is bounded away from zero and innity.

To estimate the function g we use a least squares spline estimator. If the degree of the polynomials is chosen properly, the rst or second derivatives of these estimates are piecewise

(3)

linear and lead to simple tests for positivity of these derivatives. To get the distribution of the test, we need to prove central limit theorems of the regression spline ^g(x) and its derivatives. We also provide results on the maximal deviation for some derivatives of ^g(x). In fact, these results are interesting in themselves and are formulated in Section 2. We discuss the construction and consistency of tests in Section 3. We also examine their local properties.

2 Main Results

For any two sequences of positive real numbers fa n gand fb n g, we writea n b n to mean that

an=bn stays bounded between two positive constants. Let 0 = 0 < 1 < ::: <

k+1 = 1 be a

subdivision of the interval 01] by k distinct points. We dene S(kd) as the collection of all polynomial splines of order d (degree < d;1) having a sequence of knots 1 < ::: <

k. The

classS(kd) of such splines is a linear space of functions with dimension (k+d):A basis for this linear space is provided by the B-splines (see Schumaker 1981). LetfN1::N

k+d

gdenote the set

of normalized B-splines. The least squares spline estimator of gis dened by ^ g(x) =k+d X p=1 ^ pNp(x) where ^ = ^ 1 ^k+d 0 = arg min 2R k +d n X i=1 ( Yi ; k+d X p=1 pNp(xi) )2 :

We denek byk= max0<i< k( i+1

;

i). LetN(x) be the vector of N

p(x)p= 1:::k+dand F = (N(x1):::N(xn)) andMn = 1 n n X i=1 N(xi)N(xi) 0 :

LetA(j)(x) be a the vector in R n dened by A(j)(x) = a(j) 1 (x):::a(j) n (x) 0 = 1p nF0M;1 n N (j)(x) 0< j < d ;2:

Basic least squares arguments prove that ^ = 1_nM;1 n F Y (1) whereY=(Y1:::Y n) 0

:We can also write ^ g(j)(x) ;Eg^ (j)(x) = 1 p nA(j)(x) 0 Z (2) withZ=(Z1:::Z n) 0 :

We need to specify some conditions. Here we assume that

k k

;1: (3)

Such an assumption is valid when the knots are generated by a positive continuous density on 01]:In the case where the design pointsfx

i g

n

i=1 are deterministic, we assume that

sup x201] jH n(x) ;H(x)j= ; k;1 (4) where Hn(x) is the empirical distribution function of

fx i

g n

i=1 and H(x) is the distribution limit

with positive density h(x): Notice that whenx is random we obtain from the Glivenko-Cantelli Theorem sup x201] jH n(x) ;H(x)j=O p n;1=2 :

(4)

Forx2 ;

i i+1

we dene the functionbd(:) by

bd(x) = ; g(d)( i) ; i ; i+1 d d! Bd x; i i ; i+1

whereBd(:) is the dth Bernoulli polynomial (see Barrow and Smith 1978). We also set

(j)(x) = ^ g(j)(x) ;g( j)(x) ;b (j) d (x):

Theorem 1 provides the asymptotic normality of (j)(x) for xed and random design.

Theorem 1

Letg2C

d01]: Suppose that k2j+1 =

(n) when x is deterministic and k2 j+1 =

;

n1=2

whenx is random, 0< j < d;2: Assume that(3) and (4) hold, then for all x201] p n(j)(x) q A(j)(x) 0 ;A(j)(x) N(01)

where; is the n nmatrix with (ij)thelement; ij=

i;j:

A condence band for g(j)(x) is easily obtained from Theorem 1. The next result is on the

maximal deviation of ^g(d;2)(x): It is worth noting that ^g(d;2) is a linear function between any

pair of adjacent knots i and i+1and it follows that

sup x201] ^ g(d;2)(x) = max 0< i<k+1 ^ g(d;2)( i) and inf x201] ^ g(d;2)(x) = min 0<i< k+1 ^ g(d;2)( i): (5)

Moreover ^g(d;2)is non-negative if and only if it is non-negative on the knots. This is essential for

our test procedure.

Theorem 2

Suppose that the assumptions of Theorem 1 hold. Then

P 8 < : un 0 @ max 0<i < k+1 p n(d;2)( i) q A(d;2)( i) 0 ;A(d;2)( i) ;v n 1 A < x 9 = !exp(;exp(;x)) (6) whereun= (2logn) 1=2 and vn= (2logn) 1=2 ; 1 2 (2logn);1=2 (loglogn+ log 4 ): 3 Inference

In this section we provide tests of monotonicity and convexity.

Testing convexity:

We consider the problem of testing whether the regression function is convex or not. The null hypothesis is H : g is convex and the alternative is H1 : the null is false. The idea of our test

procedure is as follows: the function g is twice di!erentiable and convex if and only if for all

xg(2)₍_x₎0or, in other words:

sup x n ;g (2)₍_x₎o <0:

Therefore, for any consistent estimator ^g of g we can expect thatP ; supx ;^g(2)(x) <0 is close to 1 wheng is convex. Then, it is natural to reject the null hypothesis of the test (that is the convexity of g) for large values of supx

;^g(2)(x)

:We already mentioned that ^g(d;2)is

non-negative if and only if it is non non-negative on the knots. So, the distribution of supx

;^g( d;2)(x)

is the same with the distribution of max

^ g(d;2)( 0):::^g(d;2) ; k+1

:Therefore, using a cubic spline estimator (d= 4), we can construct a test of convexity based on Theorem 2 as follows. We reject the convexity of g at levelwhen

un 0 @ max 0< i<k+1 ; p ng^(2)₍ i) q A(2)₍ i) 0 ;A(2)₍ i) +vn 1 A log ; 1 log(1;) : (7)

(5)

Testing monotonicity:

More precisely we consider testing whether the regression function is monotonically increasing. The testing procedure is an analogue of the convexity test. However we must use a quadratic spline estimator forg (d= 3). The null hypothesis is rejected at levelwhen

un 0 @ max 0< i<k+1 ; p ng^(1)₍ i) q A(1)₍ i) 0 ;A(1)₍ i) +vn 1 A log ; 1 log(1;) (8) where ^g(1) _{is the rst derivative of the quadratic spline estimator of} _g:

Obviously, these testing procedures can be generalized to testing the non-negativity of the (d;2)thderivative of g by using the spline of orderd:

In applications, the covariance matrix ;is unknown. Therefore, we must estimate it. The estima-tors which we shall use for ;ij=

ji;jj are ^ h= 1n n;h X i=1 ; Zi ;Z" ; Zi+h ;Z" h= 0:::n;1

where "Z is the sample mean. The estimators ^hh= 0:::n

;1, have the desirable property that

for eachn 1 the matrix ^; with elements ^; ij = ^

ji;jj is non-negative denite (cf. Brockwell

and Davis 1991).

Asymptotic power:

To make a local power calculation for the tests described above, we need to consider the behavior of di!erent statistics (calculated under a xed but unknown point g0 2 H

) for a sequence of

alternatives of the form

gn(x) =g0(x) +n'(x)

wheregn lies in the alternative hypothesis,'(:) is a known function andn is a sequence of real

variables converging to zero.

Theorem 3

We suppose that the assumptions of Theorem 1 hold and that

n(logn)

1=2

n1=2k(;2d+3)=2

!+1: (9)

Then the test (convexity whend= 4 and monotonicity whend= 3) has a power equal to one under the above local alternatives.

Our tests are asymptotically more powerful than the tests cited above. However, bootstrapping may improve considerably the power of the tests. Besides, it would be desirable to study their small sample behaviour through Monte Carlo simulations.

References

1] G.G. Agarwal and W.J Studden. Asymptotic integrated mean square error using least squares and bias minimizing splines. The Annals of Statistics, 8(6):1307{1325, 1980.

2] D.L. Barrow and P.W. Smith. Asymptotic properties of bestl201] approximation by splines

with variables knots. Quarterly of applied mathematics, pages 293{304, October 1978. 3] M.C. Bowman, A.W. Jones and I. Gijbels. Testing monotonicity of regression. Journal of

computational and Graphical Statistics, 7(4):489{500, 1998.

4] P.J. Brockwell and R.A. Davis. Times Series: Theory and Methods, Second Edition. Springer Series in Statistics, 1991.

5] U. Grenander and G. Szego. Toeplitz Forms and Their Applications. Chelsea Publishing Company, New York, 1984.

6] G. Leadbetter, M.R. Lindgren and H. Rootzen. Extremes and related properties of random sequences and processes. Springer-Verlag, 1983.

(6)

7] M. Peligrad. On the asymptotic normality of sequences of weak dependent random variables. J. Theor. Prob, 9(3):703{715, 1996.

8] W. Schlee. Nonparametric test of the monotony and convexity of regression. Nonparametric Statistical Inference, 2:823{836, 1980.

9] L. Schumaker. Spline function: Basic theory. John Wiley, New York, 1981.

10] A. Yatchew and L. Bos. Nonparametric regression and testing in economic models. J. of Quantitative Economics, 13:81{131, 1997.

11] A.J. Yatchew. Nonparametric regression tests based on least squares. Econometrics Theory, 8:435{451, 1992.

12] X. Zhou, S.Shen and D.A. Wolfe. Local asymptotics for regression splines and condence regions. The Annals of Statistics, 26(5):1760{1782, 1998.

4 Proofs

The proofs of the theorems whenx is deterministic and whenxis random use similar arguments except for the fact that in the latter case, we must write for examplevar

(j)(x) jx instead of var (j)(x)

:Hence we give the proofs for the deterministic case only.

Proof of Theorem 1: Reasoning as in proof of Theorem 1 in Barrow and Smith (1978), it is easy to see thatE

(j)(x) = ; k;d+j : We can write(j)(x) = 1 p n P n i=1a (j) i (x)Z i+ ; k;d+j :

According to Corollary 2.1 in Peligrad (1996), it suces to prove max i a (j) i (x) r nvar (j)(x) !0 asn!1 (10) and sup n 1 nvar (j)(x) n X i=1 n a(j) i (x) o2 <1: (11)

Straightforward calculations prove that var

(j) (x) = 1 nA (j)(x) 0 ;A(j)(x): This can be

rewritten is the following form

var (j)(x) = 1_n₂trh ;F0M;1 n N (j)(x)N(j)(x) 0 M;1 n F i :

Now, using the Lemma 6.5 in Zhou et al. (1998), we get

min; n tr h N(j)(x)N(j)(x) 0 M;1 n i < var (j) (x) < max_{n tr}; h N(j)(x)N(j)(x) 0 M;1 n i

where min; and max; are, respectively, the smallest and largest eigenvalues of ;:A classical

result on Toeplitz matrices (see Grenander and Szego 1984) proves thatmin;andmax;converge,

respectively, to the minimum and the maximum of the spectral density of Z:Agarwal and Studden (1980) prove that minM;1

n

k and maxM ;1 n

k: We can also prove that for each x there is

a p such that N (j) p (x) k

j: Therefore using again Lemma 6.5 in Zhou et al. (1998), we get

var

(j)(x)

k2

j+1=n:Hence (10) and (11) follow easily.

Proof of Theorem 2: We dene i andji;jj as

i= p n(d;2)( i) q A(d;2)( i) 0 ;A(d;2)( i) :

(7)

and ji;jj= corr ; ij

From Theorem 1 we know thati is asymptotically normally distributed. Therefore, according to

Theorem 6.2.1 in Leadbetter et al. (1983) it suces to prove thatnlogn

!0:We have covn (d;2)( i) (d;2) ; j o = 1_nA(d;2)( i) 0 ;A(d;2) ; j :

Using again Lemma 6.5 in Zhou et al. (1998) we obtain

covn (d;2) ( i) (d;2) ; j o2 < (max_n₂;)2n A(d;2)( i) 0 A(d;2) ; j o2 :

We can also write

A(d;2)( i) 0 A(d;2) ; j =X pq mpqN (d;2) p ( i)N (d;2) q ; j

where mpq are the elements of the matrixM ;1

n : One can easily see thatN

(d;2)

p (

i) = 0 if p < i

or p i+d and otherwise we have N (d;2) p ( i) = O ; kd;2

: On the other hand we have

jm pq j=O ; kjp;qj

for some2(01) (see Lemma 6.3 in Zhou et al.1998). Now we takej=i+n

to obtain A (d;2)( i) 0 A(d;2) ; j =O k2(d;2)+1n;d+2 : Therefore n < c1 n;d+2

which proves Theorem 2.

Proof of Theorem 3: We dene mi andqby

mi= p n g(d;2) 0 ( i) + n' (d;2)( i) +b (d;2) d ( i) q A(d;2)( i) 0 ;A(d;2)( i) q= log ; 1 log(1;) :

Then the power of the test under the local alternatives is given by

P h un max i f; i ;m i g+v n q i P h un ;max i i+ max i (;m i) +vn q i :

Hence to get a power equal to one it is enough to prove thatunmaxi( ;m

i)

!+1:Becauseg n is

non convex and the i are dense in 01] there is a postive realsuch that

max i ;g (d;2) 0 ( i) ; n'( i) > : Besides, we have max x201] b (d;2) d (x) =O ; k;d and nally p n q A(d;2)( i) 0 ;A(d;2)( i) p n k(;2d+3)=2: