Design and low-cost Design and low-cost implementation of a robust implementation of a robust multichannel noise reduction multichannel noise reduction scheme for cochlear implants scheme for cochlear implants

Design and low-cost Design and low-cost

implementation of a robust implementation of a robust multichannel noise reduction multichannel noise reduction scheme for cochlear implants scheme for cochlear implants

Simon Doclo

, Ann Spriet

, Jean-Baptiste Maj

,

Marc Moonen

, Jan Wouters

, Bas Van Dijk

, Jan Janssen

1

Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium

2

Laboratory for Exp. ORL, KU Leuven, Belgium

3

Cochlear Technology Centre Europe, Belgium

DARTS, 22 October 2003

2

Overview Overview

• Problem statement : hearing in background noise

• Adaptive beamforming: GSC

o not robust against model errors

• Design of robust noise reduction algorithm

o robust fixed spatial preprocessor o robust adaptive stage

• Experimental results

• Low-cost implementation of adaptive stage

o stochastic gradient algorithms

o computational complexity + memory requirements

• Conclusions

33

Problem statement Problem statement

• Hearing problems effect more than 12% of population

• Digital hearing instruments allow for advanced signal processing, resulting in improved speech understanding

• Major problem: (directional) hearing in background noise

o reduction of noise wrt useful speech signal o multiple microphones + DSP in BTE

o current systems: simple fixed and adaptive beamforming o robustness important due to small inter-microphone distance

hearing aids and cochlear implants

design of robust multi-microphone noise reduction scheme

Introduction

-Problem statement -State-of-the-art -GSC

Fixed beamforming

Adaptive stage

Implementation

Conclusions

4

Cochlear implants Cochlear implants

• Working principle: sound is converted to electrical stimuli in speech processor, allowing deaf people to hear again

cochlea

brain

middle ear sound

external ear

multi-channel electrode stimulator

box speech

processor

Introduction

-Problem statement -State-of-the-art -GSC

Fixed beamforming

Adaptive stage

Implementation

Conclusions

55

Algorithmic requirements Algorithmic requirements

• ‘Blind’ techniques: unknown noise sources and acoustic environment

• Adaptive: time-variant signals and acoustic environment

• Robustness:

o microphone characteristics (gain, phase, position)

o other deviations from assumed signal model (e.g. VAD)

• Implementation issues:

o number of microphones

o low computational complexity o memory

Introduction

-Problem statement -State-of-the-art -GSC

Fixed beamforming

Adaptive stage

Implementation

Conclusions

6

State-of-the art noise reduction State-of-the art noise reduction

• Single-microphone techniques:

o spectral subtraction, Kalman filter, subspace-based

o only temporal and spectral information  limited performance

• Multi-microphone techniques:

o exploit spatial information

o Fixed beamforming: fixed directivity pattern

o Adaptive beamforming (e.g. GSC) : adapt to different acoustic environments  improved performance

o Multi-channel Wiener filtering (MWF): MMSE estimate of speech component in microphones  improved robustness

Sensitive to a-priori assumptions

Robust scheme, encompassing both GSC and MWF

Introduction

-Problem statement -State-of-the-art -GSC

Fixed beamforming

Adaptive stage

Implementation

Conclusions

77

Adaptive beamforming: GSC Adaptive beamforming: GSC

• Fixed spatial preprocessor:

o Fixed beamformer creates speech reference o Blocking matrix creates noise references

• Adaptive noise canceller:

o Standard GSC minimises output noise power

Spatial preprocessing

]

0[k u

]

1[k u

]

1[k u_N Fixed

beamformer A(z) Speech

reference ]

0[k y

Blocking matrix

B(z) Noise

references ]

1[k y

]

2[k y

]

1[k y_N

Adaptive Noise Canceller

 

] [k z

]

1[k w

]

2[k w

]

1[k w_N (adaptation during noise)

]

0

[ k y

 

noise speech

] [ ]

[ ]

[ k x k v k y





 





]

[

[ ] [ ] [ ]

min E v k

k k

k

w v

w

    





T T

T T N T

T

k k

k k

k k

k k

] [ ]

[ ]

[ ]

[

] [ ]

[ ]

[ ]

[

1 2

1

1 2

1









v v

v v

w w

w w





Introduction

-Problem statement -State-of-the-art -GSC

Fixed beamforming

Adaptive stage

Implementation

Conclusions

8

Robustness against model errors Robustness against model errors

• Spatial preprocessor and adaptive stage rely on assumptions (e.g. no microphone mismatch, no reverberation,…)

• In practice, these assumptions are often not satisfied

o Distortion of speech component in speech reference o Leakage of speech into noise references, i.e.

• Design of robust noise reduction algorithm:

1. Design of robust spatial preprocessor (fixed beamformer) using statistical knowledge about microphone characteristics 2. Design of robust adaptive stage by taking speech distortion

into account in optimisation criterion  speech distortion weighted multichannel Wiener filter (SDW MWF)

Speech component in output signal gets distorted ]

0

[ k x 0 x [k ] 

] [ ] [ ]

[ ]

[ k x

k k k

z

    w

x

Limit distortion both in and x

[ k ] w

[ k ] x [ k ]

Introduction

-Problem statement -State-of-the-art -GSC

Fixed beamforming

Adaptive stage

Implementation

Conclusions

99

Design of fixed beamformer Design of fixed beamformer

• FIR filter-and-sum structure: arbitrary spatial directivity pattern for arbitrary microphone configuration

• Objective: calculate fixed FIR filters w

[k] such that

beamformer performs desired spatial and spectral filtering

Far-field:

- planar waves - equal attenuation

Spatial directivity pattern: ( , )

) (

) , ) (

,

(  





 

 w

g

S

H  Z 

Desired spatial directivity pattern: D (  ,  )

Introduction

Fixed beamforming -Broadband design -Robustness

Adaptive stage

Implementation

Conclusions

10

Design procedures Design procedures

• Design filter w such that spatial directivity pattern optimally fits  minimisation of cost function

o Broadband problem: no design for separate frequencies 

 design over complete frequency-angle region

• Cost functions:

o Least-squares  quadratic function

o Non-linear cost function  iterative optimisation = complex!

o Eigenfilter based on TLS-criterion  GEVD

 



 F   H   D   d  d  J

(w ) ( , ) ( , ) ( , )

amplitude and phase

) , (   H

) , (   D

 

 



 F   H   D   d  d  J

(w ) ( , ) ( , )

( , )

only amplitude

 



       



 H D d d

F

J

e TLS T

1 ) , ( )

, ) (

, ( )

(

2

w Q w w

Introduction

Fixed beamforming -Broadband design -Robustness

Adaptive stage

Implementation

Conclusions

1111

Non-linear procedure TLS-Eigenfilter

Simulations Simulations

Angle (deg) Freq (Hz)

dB

Angle (deg) Freq (Hz)

dB

Parameters:

-N=5, d=4cm -L=20, f_s=8kHz -Pass: 40^o-80^o -Stop: 0^o-30^o + 90^o-180^o

Delay-and-sum

Angle (deg) Freq (Hz)

dB

12

• Small deviations from assumed microphone characteristics (gain, phase, position)  large deviations from desired directivity

pattern, especially for small-size microphone arrays

• In practice microphone characteristics are never exactly known

• Consider all feasible microphone characteristics and optimise o average performance using probability as weight

– requires statistical knowledge about probability density functions

o worst-case performance  minimax optimisation problem

– finite grid of microphone characteristics  high complexity

Robust broadband beamforming Robust broadband beamforming

1 0

1 0

1

0

, , ) ( ) ( )

(

0 1









 





A A

mean

J A A f A f A dA dA

J

N









Incorporate specific (random) deviations in design







 



  





/ cos phase

) , ( gain

) , ( )

,

(

n

a e

e

A      



Measurement or calibration procedure

Introduction

Fixed beamforming -Broadband design -Robustness

Adaptive stage

Implementation

Conclusions

1313

Simulations Simulations

• Non-linear design procedure

• N=3, positions: [-0.01 0 0.015] m, L=20, f

=8 kHz

• Passband = 0

-60

, 300-4000 Hz (endfire) Stopband = 80

-180

, 300-4000 Hz

• Robust design - average performance:

Uniform pdf = gain (0.85-1.15) and phase (-5

-10

)

• Deviation = [0.9 1.1 1.05] and [5

-2

5

]

Design J J

J

J

Non-robust 0.1585 87.131 275.40 3623.6

Average cost 0.2196 0.2219 0.3371 0.4990 Maximum

cost 0.1707 0.1990 0.4114 0.4167

Introduction

Fixed beamforming -Broadband design -Robustness

Adaptive stage

Implementation

Conclusions

14

Non-robust design Robust design

No deviationsDeviations (gain/phase)

Simulations Simulations

Angle

(deg) Frequency

(Hz)

dB

Angle

(deg) Frequency

(Hz)

dB

Angle

(deg) Frequency

(Hz)

dB

Angle

(deg) Frequency

(Hz)

dB

Introduction

Fixed beamforming -Broadband design -Robustness

Adaptive stage

Implementation

Conclusions

Non-robust design Robust design

Simulations Simulations

1515

16

Design of robust adaptive stage Design of robust adaptive stage

• Distorted speech in output signal:

• Robustness: limit by controlling adaptive filter

o Quadratic inequality constraint (QIC):

= conservative approach, constraint  f (amount of leakage) o Take speech distortion into account in optimisation criterion

– 1/ trades off noise reduction and speech distortion

• 1/ = 0 or no speech leakage  GSC

• 1/ = 1  MMSE estimate of speech component in speech reference signal

– Regularisation term ~ amount of speech leakage

] [ ] [ ]

[ ]

[ k x

k k k

z

    w

x ]

[ ] [ k k

T

x

w w [k ]

 ] 

[k w

 



  ^{ }



]

[

1 [ ] [ ]

] [ ] [ ]

[

min E v k

k k E

k k

k

w v w x

w

^{   } 

noise reduction speech distortion

Limit speech distortion, while not affecting noise reduction performance in case of no model errors  QIC

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

1717

Wiener solution Wiener solution

• Optimisation criterion:

• Problem: clean speech and hence speech correlation matrix are unknown!

Approximation:

• VAD (voice activity detection) mechanism required!

 



  ^{ }



]

[

1 [ ] [ ]

] [ ] [ ]

[

min E v k

k k E

k k

k

w v w x

w

^{   } 

 ^{[ ] [ ]}   ^{[ ] [ ]}  ^{ [ ] [ ] }

] 1

[

1



 



 



 





k v k E k

k E k

k E

k x x

v v

v

w 

] [k

 ^{[ k ]} x ^{[ k ]} 

E x x

 ^{[ k ] [ k ]}   ^{E [ k ] [ k ]}   ^{E [ k ] [ k ]} 

E x x

 y y

 v v

1

1 1 ]

[



 

 



  

 



 



  

k

w ^E  ^{x [ k ] x}

^{[ k ]}  ^E  ^{v [ k ] v}

^{[ k ]}  ^{E v  [ k ] v}

^{[ k   ] }

speech-and-noise periods noise-only periods

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

18

Spatially-preprocessed SDW-MWF (1) Spatially-preprocessed SDW-MWF (1)

• In new optimisation criterion additional filter on speech reference signal may be added

]

0[k w

Spatial preprocessing

]

0[k u

]

1[k u

]

1[k u_N Fixed

beamformer A(z) Speech

reference ]

0[k y

Blocking matrix

B(z) Noise

references ]

1[k y

]

2[k y

]

1[k y_N Multi-channel Wiener Filter

(SDW-MWF)

 

] [k z

]

1[k w

]

2[k w

]

1[k w_N

]

0

[ k w

 



  ^{ }



]

[

1 [ ] [ ]

] [ ] [ ]

[

min E v k

k k E

k k

k

w v w x

w

^{   } 





T

T

k k k

k ] [ ] [ ] [ ]

[  w

w

w

w 

Speech Distortion Weighted Multichannel Wiener Filter (SDW-MWF)

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

1919

Spatially-preprocessed SDW-MWF (2) Spatially-preprocessed SDW-MWF (2)

• SP-SDW-MWF encompasses both GSC and SDW-MWF as special cases:

o No filter on speech reference 

 speech distortion regularised GSC (SDR-GSC)

– regularisation term added to GSC: the larger the speech leakage, the larger the regularisation

– special case: 1/ = 0 corresponds to traditional GSC

– SDR-GSC outperforms GSC with quadratic inequality constraint o Filter on speech reference 

 SDW-MWF on pre-processed microphone signals

– in absence of model errors = cascade of GSC + single-channel postfilter (SDW Wiener filter)

– Model errors do not effect its performance!

]

0

[ k w

]

0

[ k w

Outperforms QIC-GSC and SDR-GSC

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

20

Experimental validation (1) Experimental validation (1)

• Set-up:

o 3-mic BTE mounted on dummy head in office room (d = 1cm, 1.5cm) o Speech source in front of dummy head (90)

o 5 stationary speech-like noise sources: 75, 120, 180, 240, 285

o Microphone gain mismatch at 2

microphone

• Performance measures:

o Intelligibility-weighted signal-to-noise ratio

– Ii = band importance of i th one-third octave band

– SNRi = signal-to-noise ratio in i th one-third octave band o Intelligibility-weighted spectral distortion

– SDi = average spectral distortion in i th one-third octave band

2

i I

i

I SNR

SNR

1 intellig







i I

i

I SD

SD

1 intellig







 

f

f x

i

f

df f

i

G

c i c

, 6 / 1 6

/ 1 2

2 10

2 2

) ( log

10 SD

6 , / 1

6 , / 1



 

^G

^{( f ) } _E ^E _ ^ _X ^Z

⁽ ₍ ^f _f ⁾ ₎ ^ _

(Power Transfer Function for speech component)

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

2121

Experimental validation (2) Experimental validation (2)

• SDR-GSC:

o GSC (1/ = 0) : degraded performance if significant leakage

o 1/ > 0 increases robustness (speech distortion  noise reduction)

• SP-SDW-MWF:

o No mismatch: same as SDR-GSC, larger due to SDW-WF post-filter

o Performance is not degraded by mismatch

0 w



0 w



intellig

 SNR SD

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

22

Experimental validation (3) Experimental validation (3)

• GSC with QIC ( ) : QIC increases robustness GSC

• For large mismatch: less noise reduction than SP-SDW-MWF

 ] 

[k w

QIC  f (amount of speech leakage)  less noise reduction than SDR-GSC for small mismatch

SP-SDW-MWF achieves better noise reduction than QIC-GSC, for a given maximum speech distortion

level

Introduction

Fixed beamforming

Adaptive stage -SP SDW MWF -Experimental

validation

Implementation

Conclusions

2323

Low-cost implementation (1) Low-cost implementation (1)

• Algorithms (in decreasing order of complexity):

o GSVD-based – chic et très cher

o QRD-based, fast QRD-based – chic et moins cher o Stochastic gradient algorithms – chic et pas cher

• Stochastic gradient algorithm (time-domain) :

o Cost function

results in LMS-based updating formula

o Allows transition to classical LMS-based GSC by tuning some parameters (1/, w

)

 



 1  ^ [ ] [ ] ^



] [ ] [ ]

[ )

( E v k k k E k k

J w w

v w

x

 









 

 ^{[ ] [ ] [ ] [ ]} 

] [ ]

1

[ k w k v k v

k v

k w k

w       

regularisation term

] [ ] [ ] 1 [

k k

k x

w

 x



Classical GSC

Introduction

Fixed beamforming

Adaptive stage

Implementation -Stochastic gradient -Complexity and

memory

Conclusions

24

Low-cost implementation (2) Low-cost implementation (2)

• Stochastic gradient algorithm (time-domain):

o Regularisation term is unknown

– Store samples in memory buffer during speech-and-noise periods and approximate regularisation term

– Better estimate of regularisation term can be obtained by smoothing (low-pass filtering)

• Stochastic gradient algorithm (frequency-domain):

o Block-based implementation: improve gradient estimate by averaging over K samples

o Frequency-domain: fast convolution and fast correlation Complexity reduction

Tuning of  and 1/ per frequency

Still large memory requirement due to data buffers

o Approximations allow to replace data buffers by correlation matrices in frequency-domain  memory reduction

] [ ] [ ] 1 [

k k

k x

w

 x



Large buffer required

Introduction

Fixed beamforming

Adaptive stage

Implementation -Stochastic gradient -Complexity and

memory

Conclusions

2525

Complexity + memory Complexity + memory

• Parameters:

= 16kHz, L

= 10000

• Computational complexity:

• Memory requirement:

Algorithm Complexity MIPS

QIC-GSC (3N-1)FFT + 14N – 12 1.38

SDW-MWF (no approximation) (3M+5)FFT + 28M + 6 3.46 SDW-MWF (approximations) (3M+2)FFT + 8M

+ 14M + 3 2.8

Algorithm Memory kWords

QIC-GSC 4(N-1)L + 6L 0.64

SDW-MWF (no approximation)

+ 6LM + 7L 41.22 SDW-MWF (approximations)

+ 6LM + 7L 2.24

Complexity comparable to FD implementation of QIC-GSC

Substantial memory reduction through approximations

Introduction

Fixed beamforming

Adaptive stage

Implementation -Stochastic gradient -Complexity and

memory

Conclusions

26

Conclusions Conclusions

• Design of robust multimicrophone noise reduction algorithm:

o Design of robust fixed spatial preprocessor

 need for statistical information about microphones o Design of robust adaptive stage

 take speech distortion into account in cost function

• SP-SDW-MWF encompasses GSC and MWF as special cases

• Experimental results:

o SP-SDW-MWF achieves better noise reduction than QIC-GSC, for a given maximum speech distortion level

o Filter

improves performance in presence of model errors

• Implementations: Stochastic gradient algorithms available at affordable complexity and memory

• Further research: robustness against VAD-errors

o e.g. parameters dependent on input SNR

Spatially pre-processed SDW Multichannel Wiener Filter

Introduction

Fixed beamforming

Adaptive stage

Implementation

Conclusions

2727

Relevant publications Relevant publications

• Doclo S., Moonen M., “GSVD-Based Optimal Filtering for Single and Multi-Microphone Speech Enhancement”, IEEE Transactions on Signal Processing, vol. 50, no. 9, Sep.

2002, pp. 2230-2244.

• Spriet A., Moonen M., Wouters J., “A multichannel subband GSVD approach to speech enhancement”, European Transactions on Telecommunications, vol. 13, no. 2, Mar.

2002, pp. 149-158.

• Spriet A., Moonen M., Wouters J., “Robustness analysis of GSVD based optimal

filtering and generalized sidelobe canceller for hearing aid applications”, in Proc. of the IEEE Workshop on Applications on Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 2001, pp. 31-34.

• Maj J.B., Royackers L., Moonen M., Wouters J., “SVD-Based Optimal Filtering Technique For Noise Reduction In Dual Microphone Hearing Aids: A Real Time Evaluation”, submitted to IEEE Transactions on Biomedical Engineering.

• Doclo S., Moonen M., “Design of far-field and near-field broadband beamformers using eigenfilters”, Signal Processing, vol. 83, no. 12, pp. 2641-2673, Dec. 2003.

• Doclo S., Moonen M., “Design of broadband beamformers robust against gain and phase errors in the microphone array characteristics”, IEEE Transactions on Signal Processing, vol. 51, no. 10, Oct. 2003, pp. 2511-2526.

• Doclo S., Moonen M., “Design of broadband beamformers robust against microphone position errors”, in Proc. of the International Workshop on Acoustic Echo and Noise Control, Kyoto, Japan, Sep. 2003, pp. 267-270.

• Spriet A., Moonen M., Wouters J., “Spatially pre-processed speech distortion weighted multi-channel Wiener filtering for noise reduction”, Internal Report 03-46, ESAT-

SISTA, K.U.Leuven (Leuven, Belgium), 2003.

• Spriet A., Moonen M., Wouters J., “Stochastic Gradient based implementation of spatially pre-processed multi-channel Wiener filtering for noise reduction in hearing aids”, Internal Report 03-47, ESAT-SISTA, K.U.Leuven (Leuven, Belgium), 2003.

Available at SISTA publication engine:

http://www.esat.kuleuven.ac.be/~sistawww/cgi-bin/pub.pl

Introduction

Fixed beamforming

Adaptive stage

Implementation

Conclusions