An inequality on entropies

(1)

An inequality on entropies

Citation for published version (APA):

Brands, J. J. A. M., & Nijst, A. G. P. M. (1974). An inequality on entropies. (Memorandum COSOR; Vol. 7402). Technische Hogeschool Eindhoven.

Document status and date: Published: 01/01/1974 Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne

Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

(2)

COS

EINDHOVEN UNIVERSITY OF TECHNOLOGY Department of Mathematics

Memorandum COSOR 1974-02 Issued Februar, 1974

AN

INEQUALITY ON ENTROPIES

by

J.J.A.M. Brands and A.G.P.M. Nijst

University of Technology Department of Mathematics PO Box 513, Eindhoven The Netherlands

(3)

- 1

-In entropy theory the following inequality is well known (see Neveu [2J,

lemme 4):

Let

(PI'PZ' ••• )

be a sequence of non negative real numbers summing up to one. Then we have

00

L

_zl(Pn

) $ x + 2 I X n=1 00 with x ,-

L

n=l P log n, where n [ -X log X, X

f

0 z1(x) =

°

' , x 0

In Behara-Nath [IJ more definitions of entropy are introduced using the

set of functions z (x) = a. Remark that a. x - x a. - 1 , a. ~ 0, a.

f

1, 0 $ x $ 1 • lim z (x) = zl (x) • 0.+1 a.

In this note we shall generalize for a. ~ 1 Neveu's result mentioned

before to inequalities concerning the function z (x).

a.

Theorem. Let for a. ~

o

be g~ven

00 1

-

(..!..)a.

L

n x = _Pn a. _n=l a. Then we have 00

L

z0.+_{1(Pn )} $ (a. + 1)x + (2 + a.)IX

.

n=l a. a. Furthermore for

°

$ a. $ 1 we have 00 $ x +

2&'

a. a.

(4)

Remark that

1

-

(1.)0

lim n

₌

log n

0+0 0

and hence for o

=

0 the assertion of the theorem reduces to Neveu's result.

Lennna 1• Let - 0

-

x f (x) := 0 :::: 0 a 0

,

and let 0 ::;; p < 1 , 0 ::;; q < 1. Then we have f_{o p}(1.) f (1.) ::;; q0 q - p o q p where f (00)

=

lim f (x) a 0 x-+oo

and

=

lim f (x)

=

log x •

o

0-1-0

Proof. f is a concave function with for positive arguments a second

deriva-a

tive. Furthermore we have

lim f (x)

=

-00, f(I)

=

0, f' (l)

x+O

and lim f (x)

x-+oo

Hence for every x > 0 it follows that f (x) ::;; x - I •

0

Thus

a a 1

-

(S)- 0

f (1.) f (1.)

₌

q - p

₌

qa p ::;; q0 (.9. - 1) qa(q - p)

.

a p u. q a 0 p p

Lennna 2. Define for a :::: 0

00 S (0) := \'_L P_n

n=1

-a - n

(5)

- 3

-Proof. Let for a positive integer n be given

-x - n g (x)

:= --- =

n x Then we have g'(x) = hex log n) n 2 x 1 - e-x log n x

with hey) := (y + I)e-y - 1 •

For y > 0 we have h'(y) = -ye-y < 0 and by h(O) = 0 it follows that hey) <0

for y > O. Thus also g'(x) < 0 for x > 0 including that g (x) is decreasing

n n

for x ~ O. By definition it follows that also Sea) ~s a decreasing function

of a.

Proof of theorem. Let S > 0 and let q n- (l+8), n = 1,2, ••• , then

n 00 00

J

L

_qn $ 1 + dx ₌ +-1 n=1 1 + 8 8 1

and hence using lennna

00 00 qn - Pn 00

I

p {f (---)1 f (2-)} $

I

_pnqna $

I

( a+l _{- Pnqn)}a $ n=1 n a pn a qn n=1 Pn n=l qn 00 00 00 $

I

_n-(l+a) (1+8) _-

I

_pnqna $ _{1 +} 1

I

a 8 + a8 pnqn n= 1 n=! a + n=1 00 1

I

a 1 = + P (1 - q ) + a + 8 + as _n=1 n n a + 8 + a8 00 -a(I+8)

+ a(l + 8)

_E

_Pn - n = 1 8 + a(1 + 8)S(a(1 + 8»

.

(6)

00

L

n=1 00 p f C -I )::;

L

n a Pn n=1 SI + aS + a(1 + S)S(a(1 + S» = = co 1 _ n-a.(l+S)

L

P n - - - + n= I a a + 1 S + as + a(1 + S)S(a(1 + S» = 1 = (1 + a)(1 + S)S(a(1 + S» + a + S + as •

It follows by lemma 2 that

00 LPnfa

(i-) ::;

n=1 n 1 S(a) + (a + S + as)S(a) + a + S + as ~ ::; (1 + a)S(a) + S(1 + a)S(a) I Choosing f3 = - - - we get

Is

(a) 1 + -S 00

L

Pnfa(i-) ::; (a + I)S(a) + (a + 2)/S(a) •

n=1 n

Furthermore for 0 ::; a ::; 1 we have

s~nce

S(a) ::;

l

that

a

hence there exists a S ~ 0 such that

a + S + as

For this S it holds that

I

~

fa

~

a and

Is

(a)

00

L

Pnfa(i-)::; S(a) + (a + S + as)S(a) + +

~

+ as

n= 1 n a

and the result of the theorem has been proven.

References

S

(a) +

2/s

(a)

[I J Behara M. and Nath. P. (1973), "Additive and Non-Additive Entropies of

Finite Measurable Partitions", Lecture Notes in Mathematics 296, Probability and Information Theory II, 102-138.

L2J Neveu J. (1967), "Atomes Conditionnels d 'Espaces de Probabilite de

l'Information", Lecture Notes in Mathematics 31, Symposium on Probability Methods in Analysis, 256-271.