The covariance matrix of a multivariate locally best unbiased estimator

estimator

Citation for published version (APA):

Sander, P. C. (1979). The covariance matrix of a multivariate locally best unbiased estimator. (TH Eindhoven. THE/BDK/ORS, Vakgroep ORS : rapporten; Vol. 7903). Technische Hogeschool Eindhoven.

Document status and date: Published: 01/01/1979 Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

,

ARW

16

BDK

7903

The covariance matrix of a multivariate locally best unbiased estimator

by Peter C. Sander

Eindhoven University of Technology Netherlands

April 1979

The covariance matrix of a multivariate locally best unbiased estimator

Barankin (1949) has given the variance of a univariate locally best unbiased estimator. In this note the covariance matrix is given of a multivariate locally best unbiased estimator. The Cramer-Rao and the Chapman-Robbins-Kiefer bounds appear as special cases.

1. INTRODUCTION

Let X be a random variable defined on a measure space

(X, A).

Let p = {p(.le)le E ~} be a family of probability densities of X with

respect to the sigma finite measure ~; ~ is an index g (e) : = (g 1 (e), .•• , g (e» t wi th g. : ~ -+ lR (i

=

1,

m ~

Estimates of gee) are denoted by o(x)

=

(0

1 (x), .•. ,

set. Define ••• , m). o (x»t with

m

o. : X -+ lR (i= 1, ••• , m) ~-measurable; 0 (X) is the corresponding

~

estimator.

The class of unbiased estimators o (X) of gee) is denoted by Z,

while Z(80), eO

E~,

denotes the subclass of Z with the property

E(o~(x)leO)

< 00. (i

=

1, ... , TIl).

~

Definition 1. oO(X) is a locally best unbiased estimator (LBUE) of

g (8) ';n _~

e

O, . _~ f _{t e o} h f 1 1 " _{ow~ng lS true} f _{or every a}

₌

( _{aI' ••• ,} a ) _m t

E lR m

and

ii) var ([ a.o.(x)le )

m o o

~ var ([ a.o.(x)le ) for alIa

m

°

E Z.

1 ~ ~ 1 ~ ~

o

Let the covariance matrix of O(X) E Z with respect to p(.le) be given by

[o(e) =

«cr~

.(e»). If oO(X) E Z(80), then oO(X) is a LBUE iff

1.J m (1) [ m L i,j=l

o

°

m

a.a.cr .. (e) for all a E lR , 0 E Z.

i,j=l ~ J ~J

°

F _{ormu a} 1 ( 1) _{means [}

° (

_e 0) _{- [}

° (

_e 0 ) . _{~s pos~t~ve sem~-}. . . d _e f' . _{~n~te, i.e.}

°

LO (eO) ~ [o(eO).

The following results are of immediate use (cf. Rao (1973) pp. 317-318).

AMS 1970 subject classifications. Primary 62FlO; Secondary 62H99.

Key words and phrases. Locally best unbiased estimator, covariance matrix, Cramer-Rao bound.

Theorem 1.

°

Then 0 (X)

°

cov(n (X),

Assume m = 1 and oO(X) E z(eO).

is a LBUE of g(e) in eO iff

o

(X) leO) =

°

for all 6(X) E Z(eO). Corollary 1

i) 6(X) is a LBUE of gee) in eO iff 0i(X) is a LBUE of gi(e) in eO (i = I, ... , m)

°

ii) if there are two LBUEs of g(e) in e , they are the same except for

°

a set of p(.le ) measure zero.

o

From this corollary it follows that LBUEs can be constructed co-ordinate wise, following Barankin (1949).

2. LINEAR TRANSFORMATIONS

2 .

Let L (X, v) be the family of all square integrable real valued functions with respect to a a-finite measure v on X, with inner product

and norm (f, g) :=

f

f g X Ilfll = (f, f)!. 2 d v for f, gEL (X, v),

By Riesz'representation theorem with every bounded linear functional H

O L

2

(X, v) + lR there corresponds an adjoint function hO E L2(X, v),

such that

(2) HO(g) =

f

g hO dv, for all g E L2(X, v).

X

Let H(g) := (HI (g), ••• , Hm (g» t with Hi (i = I, ••• , m) a bounded linear, functional with adjoint hi E L2(X; v). Then we will call H : L2(X, v) +

lR~

a bounded linear functional as well, with adjoint Ht

:=

(hi' ••• , h_m).

~dr:

.,~

'.,

.l

2 m

Lemma 1. Let H : L (X, v) + 1R be a bounded linear functional with d · · t ( h ) a JOLnt H

=

hI' ... , m' Define S :

=

«

f

h. h . dv ) )

~

. I L J L ,J= X Then

ii) if there is an m x m matrix A with

then

A ~ S.

Proof

i) Let a € 1Rm. By Schwarz' inequality

(4) Hence a

ts

a

J

Ix

12

dv

~

X H). I f (3) then D: a.

f

xh. dv]2 for all x E L2(X, v) • L l. X

for all x E L2(X, v),x =f 0 and all a E 1Rm.

From (4) it follows that for an arbitrary but fixed a E 1Rm and

m x

=

1::

i=l

a.h.

II

x

112

a

ts

a = a H(x)[H(x) ] a. t t Hence, for this choice of x,

(6) a A a - a S a t t ~ O.

Since (6) is true for all a ~ lRm the proof is complete.

D

Notes

I. Since S

=

«

f

h.h.dv»~

. I' it is logical to denote S by HHt. ~ J ~ ,J=

X

This will be done in what remains.

2. If a matrix S satisfies i) and ii) of lemma 1, the following notation will be used

D

2 .1

Let K be a linear subspace of L

(X,

v) and K the orthogonal complement of K in L 2 (X, v). Let F : K -+ lR m be a bounded linear functional.

Define the bounded linear extension F of F by

e

(7)

Lemma 2. Let K be a linear subspace of L2(X, v) and let F

a bounded linear functional with extension F .

m K -+ lR be e Then Proof By lemma

Especially

Assume there is an m x m matrix T such that

Then

and hence, since Ilxll = Ilx} + x211

~

Ilx}11 (Xl E K, x2 E Kl.),

The result follows from lemma 1. D

3. THE COVARIANCE MATRIX OF A LBUE Define the measure v by

v(A)

=

f

p(.leO)d~

for all A EA.

A Assume from now on:

I. p(xle)/p(xleO) 18 defined

v -

a.e. for all e E Q and this quotient is denoted by TI(xle)

II.TI(.ls) E L2(X, v) for all

e

E Q •

2

Define K as the linear subspace of L (X, v) spanned by finite linear combinations of the TI(.le), e E Q.

Lemma 3. Let F : {~(.Ie) Ie € Q} + IRm be defined by F(~(.le» ; gee).

Then the propositions i) and ii) are equivalent:

i) there is a bounded linear transformation A L 2 (X, \) + IR m such that A(n(.le»; F(n(.IS»

ii) there is a positive semidefinite matrix M such that

(8)

for all n € :IN, a = (aI' •.• , an) E IR nand e 1 ,

...

,

n .

Furthermore, let z ; E a.~(.le1) and define ~ 1 1 m K + IR by (9) ~ (z) n . := E a.F(~(.le1». ] 1

iii) I f ii), then

e

is well-defined, Furthermore MO (8) and M ;::: MO' Proof i) ~ ii). Choose M

=

AAt, 'V n 'V

I

"~J' iii) Assume z = E a. ~(,

6 )

] J 'V and z = z. Then by (8)

hence ~ is well-defined. Since ~ is a bounded linear transformation the results follow from lemma 2,

ii) ~ i). ~ meets the requirements of i),

e

o

We are now able to give the covariance matrix of a multivariate locally best unbiased estimator.

Theorem 2. Let ¢, ¢e and M

O be defined as in lemma 3. Then i) Z(¢O) f

0

iff

(10)

f or a 11 n E IN, a = ( aI' ••• , an ) E lR, n

e

1 , ••• , e n E Q.

°

0

°

ii) I f 8 (x) E Z (e ) then I (e ) ~ HO.

iii)Z(eo) f

0

iff there is one and only one (p(.leo)

a~e.)

°

0 80

°

8 (x) E z(e ) with. I (e)

=

MO. Proof

i) _{Assume 8(x)} E. Z(e ) (the "only if" part). Then,

o

.

~n an obvious

notation, Hence (11 ) n . ~ I a.g(e ) 1 ~

=

J

X n . 8 I a·7f(.le~)d\). 1 ~ t n i u i t

u (I a.g(e »(I a.g(e » u

1 ~ ~

=

ute

f

8

f

ai7f(·le~)d\»(

f

X X n . 2

III

a·7f(.le~)11 1 ~

From lemma 3 it follows that (10) ~s true for M = MO'

Now the "if" part. By Riesz' representation theorem there is a 0° := (0°1' ... , oO)t with o? E L2(X, v) (i = I, ..• , m) such that

m ~

(I 2) <p (f)

e

Since <p (TI(.IS»

=

g(S) from (12) it follows that oO(X) E Z(SO).

e

ii) This follows immediately from (II) and lemma 3.

iii) If Z(SO) f

~

then 0° defined by (12) meets the requirements. The unicity is a result of corollary I.

4. THE CRAMER-RAO BOUND

The Cramer-Rao bound is a special case of theorem 2. This can be seen from what follows.

Theorem 3. Assume i) Z(SO) f ~

ii)

n

c lR k, Q is open

iii) p(xIS) f

°

~-ae for all SEQ

D

t t

iv) if S

=

(SI' ••• , Sk) E Q, S(i,h):= (SI' ... , Si-I' Si+h , Si+I' ' ' ' , Sk)

v) vi)

and h, a I' .•. , a_k E lR, then

lim

f {

~

a. (TI(.IS) - TI(.IS(i, h»)/h}2 dv h+OX I ~

=

f· {

lim

~

a. (TI(.IS) - TI(.IS(i, h»)/h}2 dv

X h+O I ~ M

O is positive definite

F :=

«

f

(oTI(.ls)/os.)(aTI(.ls)/oS.)dv»k

and

G :=

are well-defined and F is non singular. Then

Proof. By theorem 2

for all n E IN, a

=

(a

J, Especially for a and

e

l ,

(13) sup

m

UEm

Note that there is an

...

,

...

,

a)

E m

n ,

e

1 , ... , en

En,

u

E mm.

n en fixed t m x m matrix B such that MO

=

B B.

Define n . 1 V := 2: a.g(e ) 1 1 P := (B-1)t v g := Bu.

From

it follows that

hence

(14) sup utvvtu/(utMOU) vtM~1 v.

u

From (13) and (14) it follows that

n i t -1 (2: a.g(e» MO 1 ~ n . (2: a.g(8~» ::::; 1 ~

for. all a = (aI' •• " a ) E lR n . and e 1 ,

n

In particular

...

,

e

n r.

E .. ,

k 0 0 . t -J k 0 0

(l; a.[g(6 ) - g(e (1,h»J/h) MO (.2: a_i[g(6 ) - g(e (i,h»J/h)

i=l ~ . . ~=I

::::; II

Hence with iv)

k

15) (2: a.

1 ~

t

=

a F a

Let I be the identity matrix of order n, and B an arbitrary m x k

n

From (15) it follows that

Gt M-I

_o

G :s:;, F

hence

The result now follows from (16).

5. FINAL REMARKS

In lemma 3 MO

~s

defined by MO =

~e~et,

in other words

(17) sup

n,a,€l

It is easily seen that (17) is equivalent to

(18)

with

t [E

1g(V)-E2g(V)][EIg(V)-E2g(V)]

MO = sup, "'I""'2

f { f

p (x

I

v)d["'l (v) -"'2 (v) J) 2

/p

(xl eO)d" (x) X Q E.g(V) 1.

=

f

G g(v) d w. (v) ~

o

and in which the supremum 1.S taken over all measures wI and w

2 with wI ~ w

2 and such that the integrand in the denumerator is defined

lJ - a.e . •

(g(eo)-E g(V»(g(eO)-E g(V»t 8

°

o o · 00 E (8 ) ~ sup ----..;:~---;::"--....;~--2 00 E p(XIV) E [_00...----,,:_ ] eO p(xleO) - 1

This last result is a logical generalization of the well-known one-dimensional Chapman-Robbins-Kiefer theorem (cf. Chapman and Robbins (1951) and Kiefer (1952».

REFERENCES

[IJ Barankin, E.W. (1949). Locally best unbiased estimates, Ann. Math. Statist. 20 477-501

[2J Chapman, D.G. and Robbins, H. (1951). Minimum variance estimation without regularity assumptions.

Ann. Math. Statist. 22 581-586

[3J Kiefer, J. (1952). On minimum variance estimators.

Ann. Math. Statist. 23 627-632

[4J Rao, C.R. (1973). Linear statistical inference and its applications (second edition). Wiley, New York.

Department of Industrial Engineering Eindhoven University of Technology 5600 MB Eindhoven