A unification of adaptive A unification of adaptive multi-microphone noise multi-microphone noise reduction systems reduction systems

(1)

A unification of adaptive A unification of adaptive

multi-microphone noise multi-microphone noise

reduction systems reduction systems

Ann Spriet

^1,2

, Simon Doclo

¹

, Marc Moonen

¹

, Jan Wouters

²

1Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium

2ExpORL, Dept. Neurosciences, KU Leuven, Belgium

IWAENC-2006, 12.09.2006

(2)

22

Overview Overview

• Signal-dependent multi-microphone noise reduction

o LCMV, TF-LCMV, SDW-MWF, soft-constrained beamformer

• Definition of general cost function

• Derivation of existing + novel algorithms

• Theoretical work, no focus on implementation issues

and simulation results

(3)

3

• Signal model:

• Output signal:

Multi-microphone noise reduction Multi-microphone noise reduction

( ) f 

^s

( ) f 

ⁿ

( ) f

X X X

 

S f

 

s f

  H X1 f

 

X2 f

 

XM f



 

W f1

 

Z f W f2 

 

WM f

single speech source

 

1

 

(

( )

^s

) ( )

s

f  f S f 

s

f X

s

f

X H H 

transfer functions TF-ratio

H H s H n

Z  W X W X   W X Objectives: 1. noise reduction:

2. limit speech distortion:

H n

 0 W X

H s s

 D W X

• Reference signal : source , microphone , output FB D

^s

S X

₁^s

Multi-microphone noise reduction

General cost function

Conclusions

(4)

44

• Second-order statistics: speech and noise correlation matrix

• TFs sometimes approximated by steering vector using free- field propagation model (delay, near-field, microphone chars)

Multi-microphone noise reduction Multi-microphone noise reduction



^,



n

 E

n n H

R X X

 

1

, ,

s s s H s s s H

E P

X

 

R X X H H  

 ( )

H  d p  with p position of source with respect to microphone array

• Distributed speech source

(region P): ( ) X

₁

( )

 

p P

X d p  p

[Yermeche 2004]

Conclusions

(5)

5

Multi-microphone noise reduction Multi-microphone noise reduction

LCMV

• minimize output noise energy

• hard constraint: no distortion

• a-priori model (free-field)

[Frost 1972,

Griffiths-Jim 1982, Buckley 1986]

soft-constrained beamforming

• soft constraint: trade-off output noise energy vs. speech distortion

• a-priori model (speech region)

[Nordholm, Dam, Grbíc, Low, 2002 -2005]

TF-LCMV

• minimize output noise energy

• hard constraint: no distortion

• on-line model: estimate TF-ratios

[Gannot 2001, Hoshuyama 1999, Herbordt 2003]

SDW-MWF

• soft constraint: trade-off output noise energy vs. speech distortion

• on-line speech model

[Doclo 2002, Spriet 2004]

a-priori model vs. on-line estimation

Conclusions

soft vs. hard constraint

(6)

66

• Trade-off output noise energy and speech distortion

o on-line estimation

o based on a-priori knowledge (model, calibration)

General cost function General cost function



²



²



²



1

(1

( ) = )

s H s

m

H n H n

s H

m

s

E D

J

 E D 

 

 





 

W R W W

X W

R

X W

W W

output noise energy

speech distortion

(on-line) speech distortion (a-priori)

Conclusions

• Different signal-dependent algorithms:

o on-line estimation vs. a-priori knowledge

o hard constraint (₁₂=)  signals in speech subspace undistorted, noise suppresion in subspace orthogonal to speech subspace

vs. soft-contraint (₁₂)  spectral filtering of desired speech

• On-line noise estimation (=0)  focus on speech model (

1

,

2

)

(7)

7

General cost function General cost function



²

  

1

2

( ) =

^H ⁿ ^s ^H ^s 2 _m^s ^H ^s_m

J W W R W   E D  W X   E D  W X

Conclusions

Model hard/soft

constraint Technique

A-priori

₁=0 ₂=

LCMV

₁=0 ₂

soft-constrained beamforming

On-line

₁= ₂=0

TF-LCMV

₁ ₂=0

SDW-MWF

Combination

₁ ₂=

SDR-GSC

₁ ₂

soft-constrained SDW-MWF

(8)

88

• assumptions about speaker location, acoustics, microphones

 performance affected when assumptions are violated

A-priori speech model (

A-priori speech model (  _

₁₁

=0 =0 ) )



²



( ) =

^H ⁿ 2

E

_m^s ^H _m^s

J W W R W   D  W X

• LCMV (

₂

=): hard constraint

o speech source using free-field progation model:

o reference signal:

o solution:

( )

,1

s s s

m



m

X

m

X d p 

,1

s s

m m

D  X

1 , 1

s

/

s H s

n n

 

 d d

W R   R d 

• Soft-constrained beamforming (

₂

):

o soft constraint on (partially) modelled speech distortion term:

model for spatial characteristics, on-line estimation of spectrum o speech source in region P:

o reference signal:

o solution:

( )

,1

( )

s s s

m

X

m

 

p P

X d p  p

,1

( )

s s

m m

D X

 

p P

p



ⁿ

^

²

^P

^X^s¹ ^ ^s

^{( )}

^{s H}^,

^{( )} 

^¹

^

²

^P

^X^s¹ ^ ^s

^{( )}

  

p P

d p d p 

p P

d p

W R   

on-line a-priori

- a-priori model - on-line model - combined model

Conclusions

(9)

99

• reference signal = microphone signal

• typically requires VAD + noise more stationary than speech

 performance affected by VAD errors and non-stationary noise

On-line speech model (

On-line speech model (  _

₂₂

=0 =0 ) )



²



( ) =

^H ⁿ 1

E D

^s ^H ^s

J W W R W    W X

1

s s

D  X

Conclusions

• TF-LCMV (

1

=): hard constraint

o output speech component = speech component microphone signal

o on-line estimate of TF-ratio using non-stationarity of speech o adaptive implementation: TF-GSC, adaptive blocking matrix

,

1 1

1

^s

/

^{s H}

H s

n n

  s

   H

W H  W R  H  R H 

[Gannot 2001]

(10)

1010

• reference signal = microphone signal

• typically requires VAD + noise more stationary than speech

 performance affected by VAD errors and non-stationary noise

• SDW-MWF (

1

):

o soft constraint on speech distortion

o on-line estimate of speech correlation matrix using VAD

o SDW-MWF = TF-LCMV + single-channel postfilter  spectral filtering

On-line speech model (

On-line speech model (  _

₂₂

=0 =0 ) )



²



( ) =

^H ⁿ 1

E D

^s ^H ^s

J W W R W    W X

1

s s

D  X



ⁿ

^

¹ ^s



^¹

^

¹ ¹^s

 

W R R r

Conclusions

(11)

11

• Combination of a-priori knowledge and on-line estimation

 increase robustness against estimation errors

Combined speech model Combined speech model



²

  

1

2

( ) =

^H ⁿ ^s ^H ^s 2 _m^s ^H ^s_m

J W W R W   E D  W X   E D  W X

• SDR-GSC (

₂

=)

o combination of LCMV beamformer and SDW-MWF

o hard constraint imposed through GSC-structure (FB + BM)

o soft constraint: on-line estimated speech distortion between speech component in speech reference and output signal

[Spriet 2004]

q a

 

W W B W



^s ^H ^s ²

 ^

^a^H ^H ^s ^{s H}^, ^a

^

E D  W X  E W B X X B W

Conclusions

• Soft-constrained SDW-MWF (

₂

):

o combination of SDW-MWF and soft-constrained beamformer o speech model:

– partially updated based on incoming data

– partially computed a-priori using model or calibration data

(12)

1212

• General cost function:

o output noise energy + speech distortion o on-line estimated vs. a-priori model

o speech distortion: hard vs. soft constraint

• Derivation of signal-dependent multi-microphone noise reduction algorithms:

o LCMV, TF-LCMV, SDW-MWF, soft-constrained beamformer, SDR-GSC, soft-constrained SDW-MWF

• Extensions and combinations:

o a-priori noise model: fixed beamforming

o combination of on-line and a-priori noise model: e.g. sensitivity- constrained GSC

o several other possibilities possible!

• Combination of a-priori knowledge and on-line estimation of both speech and noise terms anticipated to enhance robustness

Conclusions Conclusions

Conclusions