Extension of the multi-channel Extension of the multi-channel Wiener filter with localization cues Wiener filter with localization cues for binaural noise reductionfor binaural noise reduction

Extension of the multi-channel Extension of the multi-channel

Wiener filter with localization cues Wiener filter with localization cues

for binaural noise reduction for binaural noise reduction

Simon Doclo

, Rong Dong

, Thomas J. Klasen

, Jan Wouters

, Simon Haykin

, Marc Moonen

1Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium

2Adaptive Systems Laboratory, McMaster University, Canada

3 Laboratory for Exp. ORL, KU Leuven, Belgium

WASPAA-2005, Oct 17 2005

Overview Overview

• Binaural hearing aids: noise reduction and preservation of binaural cues

• Overview of binaural noise reduction algorithms

• Binaural multi-channel Wiener filter:

o Estimate of speech component at both hearing aids

o Speech cues are preserved – noise cues may be distorted

• Preservation of binaural noise cues:

o Partial estimation of noise component o Extension with ITD and ILD cost function

• Experimental results

• Audio demonstration

Problem statement Problem statement

• Hearing impairment  reduction of speech intelligibility in background noise

o Signal processing to selectively enhance useful speech signal o Many hearing impaired are fitted with hearing aid at both ears

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions



P_L P_R

ITD

ILD

R

P P







• Binaural auditory cues:

o Interaural Time Difference (ITD) – Interaural Level Difference (ILD) o Binaural cues, in addition to spectral and temporal cues, play an

important role in binaural noise reduction and sound localization

Problem statement Problem statement

• Bilateral system:

o Independent processing of left and right hearing aid

o Binaural cues may be distorted (cf. poster Tim Van den Bogaert)

• Binaural system:

o Cooperation between left and right hearing aid (e.g. wireless link) o Assumption: all microphone signals are available at the same time

Objectives/requirements for binaural algorithm:

1. SNR improvement: noise reduction, limit speech distortion

2. Preservation of binaural cues (speech/noise) to exploit binaural hearing advantage

3. No assumption about position of speech source and microphones

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Binaural noise reduction techniques Binaural noise reduction techniques

• Fixed beamforming: spatial selectivity + binaural speech cues

o Maximize directivity index while restricting ITD error o Superdirective beamformer using HRTFS

broadside array, limited performance, known geometry

[Desloge, 1997]

[Lotter, 2004]

• CASA-based techniques

o Computation and application of (real-valued) binaural mask based on binaural and temporal/spectral cues

mostly for 2 microphones, “spectral-subtraction”-like problems

[Kollmeier, Peissig, Wittkop, Dong, Haykin]

• Adaptive beamforming: based on GSC-structure

o Divide frequency spectrum: low-pass portion unaltered to preserve ITD cues, high-pass portion processed using GSC

low-pass: no noise reduction, high-pass: no cue preservation o TF-GSC: minimize output energy, constraint: output speech

component = speech component in reference microphone signal binaural noise cues may be distorted

[Welker, 1997]

[Gannot, 2001]

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Binaural noise reduction techniques Binaural noise reduction techniques

• Binaural multi-channel Wiener filter

o MMSE estimate of speech component in reference microphone signal at both ears

speech cues are preserved, no assumptions about position of speech source and microphones

noise cues may be distorted

[Doclo, Klasen, Wouters, Moonen]

Extension of MWF:

preservation of binaural speech and noise cues without comprimising noise reduction performance

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Binaural multi-channel Wiener filter Binaural multi-channel Wiener filter

• Configuration: microphone array at left and right hearing aid

0,_m( ) = 0,_m( ) 0,_m( ), = 0 0 1 Y







0,0 0, 0 1 1,0 1, 1 1

( ) =











Y  

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

0( ) = 0^H( ) ( ), 1( ) = 1^H( ) ( ) , ( ) = 0^T( ) 1^T( ) ^T Z



















• Use all available microphone signals to compute output signal at

both ears  computation of filters W

and W

Binaural multi-channel Wiener filter Binaural multi-channel Wiener filter

• SDW-MWF: estimate speech component in reference mic signal at both ears + trade-off noise reduction and speech distortion

2 2

,0 {| 0, 0 0^H | } ( 0 1) {| 0^H | }

SDW r

J  E X W Y 



r

r

,0 ,1

( ) = ^{H H H}

SDW SDW SDW

J W J  J  P W RW W r r W  W_SDW = R r^1

0 0

0 1

1 1

= , =

, =

,

x M x v

P P P





    

           

r R R 0

r R R R R

r 0 R R

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Preservation of binaural cues Preservation of binaural cues

• SDW-MWF: binaural speech cues are generally preserved, binaural noise cues may be distorted

• Partial estimation of noise component

o Estimate of sum of speech component and scaled noise component

considerable reduction of noise reduction performance

[Klasen, 2005]

2 ,0( 0) {| ( 0,0 0 0, 0) 0^H | }

MSE r r

J W  E X  V W Y

• Extension of SDW-MWF with binaural cues

o Add term related to ITD and ILD cue to SDW cost function

o Link computation of filters W0 and W1

o Weight factors can be frequency-dependent

o Task: perceptually relevant expressions for binaural cues

( ) = ( )

tot SDW

J W J W

 

 

( )

( )

ITD

out des

J

ITD ITD

W

W

2 ( )

( )

ILD

out des

J

ILD  ILD

W

W

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

ITD-ILD cost functions ITD-ILD cost functions

• ITD: phase of cross-correlation between two signals

o Output:  input:

o Cost function: cosine of phase difference between cross-correlations

*

0 1 0 1

{ _{v v} } = ^H _v

E Z Z W R W _{0, 1, 0 1}

0 1

= { _r *_r } = _v( , ) s E V V R r r

0 1 0 1

2 2 2 2

0 1 0 1

( ) ( )

( ) = 1 cos( ( )) = 1

( ) ( )

H H

R v R I v I

ITD H H

R I v R v I

s s

J





 

 

W R W W R W

W W

W R W W R W

• ILD: power ratio of two signals

o Output:  input:⁰ ₂^{2 0 0}

1 1 1

{| | }

= .

{| | }

H

v v

H

v v

E Z E Z

W R W W R W

2

0, 0 0 0

2

1,1 1 1

{| | } ( , )

{| | } = ( , )

r v

r v

E V r r

P  E V R r r R

2

0 0

1 1

( ) =

H v

ILD H

v

J





  

 

W R W W W R W

• Other possibility: specifiy desired angle 

and use HRTFs

• Estimate noise cross-correlation/power during noise segments

2 2

*

0 1 0 1

( ) HRTF ( , ) HRTF ( , ),

HRTF ( , ) / HRTF ( , )

s

     

    

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Experimental results Experimental results

• Binaural recordings on KEMAR, 2 microphones on each hearing aid (d=1cm)

• Speech source in front, multi-talker babble noise at 45

• SNR=0 dB, f

=16 kHz, FFT-size N=256, 

= 

=1

• Performance measures:

o SNR improvement (left/right)

o Mean ITD/ILD cost function (speech/noise component)

Noise 1

Reverberation time

= 125 msec

HA

HA

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Experimental results Experimental results

• Partial estimation of noise component

o ITD and ILD cost function of speech and noise components decrease

o SNR improvement is significantly reduced

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0 2 4 6 8 10 12



SNR [dB]

Left ear Right ear

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

-4 -3.5 -3 -2.5 -2 -1.5 -1 -0.5 0

 J ITD [dB]

Noise component Speech component

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

Experimental results Experimental results

• Extension with ITD-ILD cost function

o ITD and ILD cost function of noise component decrease o SNR improvement is practically not reduced

o ITD and ILD cost function of speech component not degraded

1 0

2 x 10^-3

0 0.5

1 -2.5

-2 -1.5 -1 -0.5 0

 ITD cost function - noise component

 J ITD [dB]

1 0

2 x 10^-3

0 0.5

1 8.5

9 9.5 10 10.5

 SNR improvement right ear



SNR [dB]

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions

• PS: Other demos available (=5, 2 noise sources, T =250 ms)

Audio demo Audio demo

Algorithm Sound files

ITD

ILD

Input

SDW-MWF

-1.86

-1.31

Partial estimation (=0.8)

-3.40

-2.73

ITD-ILD xcorr (=0.002,=1)

-1.80

-1.41

• Speech enhancement for binaural hearing aids:

o SNR improvement

o Preservation of binaural speech and noise cues

o No assumptions about position speech source and microphones

• Suitable noise reduction algorithms:

o Multi-channel Wiener filter (but also e.g. Transfer Function GSC) speech cues are preserved noise cues may be distorted

• Preservation of binaural noise cues:

o Extension with ITD and ILD cost functions

o Other possible extensions: Interaural Transfer Function (ITF)

• Future work:

o Multiple noise sources ?

o Better perceptual cost functions/performance measures o Tuning of frequency-dependent weight factors

Conclusions and future work Conclusions and future work

Problem statement

Binaural noise reduction

Multi-channel Wiener filter

Preservation of binaural cues

Experiments

Conclusions