A unification of adaptive A unification of adaptive
multi-microphone noise multi-microphone noise
reduction systems reduction systems
Ann Spriet
1,2, Simon Doclo
1, Marc Moonen
1, Jan Wouters
21Dept. of Electrical Engineering (ESAT-SCD), KU Leuven, Belgium
2ExpORL, Dept. Neurosciences, KU Leuven, Belgium
IWAENC-2006, 12.09.2006
22
Overview Overview
• Signal-dependent multi-microphone noise reduction
o LCMV, TF-LCMV, SDW-MWF, soft-constrained beamformer
• Definition of general cost function
• Derivation of existing + novel algorithms
• Theoretical work, no focus on implementation issues
and simulation results
3
• Signal model:
• Output signal:
Multi-microphone noise reduction Multi-microphone noise reduction
( ) f
s( ) f
n( ) f
X X X
S f
s f
H X1 f
X2 f
XM f
W f1
Z f W f2
WM f
single speech source
1
(
( )
s) ( )
s
f f S f
sf X
sf
X H H
transfer functions TF-ratio
H H s H n
Z W X W X W X Objectives: 1. noise reduction:
2. limit speech distortion:
H n
0 W X
H s s
D W X
• Reference signal : source , microphone , output FB D
sS X
1sMulti-microphone noise reduction
General cost function
Conclusions
44
• Second-order statistics: speech and noise correlation matrix
• TFs sometimes approximated by steering vector using free- field propagation model (delay, near-field, microphone chars)
Multi-microphone noise reduction Multi-microphone noise reduction
,
n
E
n n HR X X
1, ,
s s s H s s s H
E P
X
R X X H H
( )
H d p with p position of source with respect to microphone array
• Distributed speech source
(region P): ( ) X
1( )
p PX d p p
[Yermeche 2004]
Multi-microphone noise reduction
General cost function
Conclusions
5
Multi-microphone noise reduction Multi-microphone noise reduction
LCMV
• minimize output noise energy
• hard constraint: no distortion
• a-priori model (free-field)
[Frost 1972,
Griffiths-Jim 1982, Buckley 1986]
soft-constrained beamforming
• soft constraint: trade-off output noise energy vs. speech distortion
• a-priori model (speech region)
[Nordholm, Dam, Grbíc, Low, 2002 -2005]
TF-LCMV
• minimize output noise energy
• hard constraint: no distortion
• on-line model: estimate TF-ratios
[Gannot 2001, Hoshuyama 1999, Herbordt 2003]
SDW-MWF
• soft constraint: trade-off output noise energy vs. speech distortion
• on-line speech model
[Doclo 2002, Spriet 2004]
a-priori model vs. on-line estimation
Multi-microphone noise reduction
General cost function
Conclusions
soft vs. hard constraint
66
• Trade-off output noise energy and speech distortion
o on-line estimation
o based on a-priori knowledge (model, calibration)
General cost function General cost function
2
2
2
1
(1
( ) = )
s H s
m
H n H n
s H
m
m
s
E D
J
E D
W R W W
X W
R
X W
W W
output noise energy
speech distortion
(on-line) speech distortion (a-priori)
Multi-microphone noise reduction
General cost function
Conclusions
• Different signal-dependent algorithms:
o on-line estimation vs. a-priori knowledge
o hard constraint (12=) signals in speech subspace undistorted, noise suppresion in subspace orthogonal to speech subspace
vs. soft-contraint (12) spectral filtering of desired speech
• On-line noise estimation (=0) focus on speech model (
1,
2)
7
General cost function General cost function
2
1
2
( ) =
H n s H s 2 ms H smJ W W R W E D W X E D W X
Multi-microphone noise reduction
General cost function
Conclusions
Model hard/soft
constraint Technique
A-priori
1=0 2=
LCMV
1=0 2
soft-constrained beamforming
On-line
1= 2=0
TF-LCMV
1 2=0
SDW-MWF
Combination
1 2=
SDR-GSC
1 2
soft-constrained SDW-MWF
88
• assumptions about speaker location, acoustics, microphones
performance affected when assumptions are violated
A-priori speech model (
A-priori speech model (
11=0 =0 ) )
2
( ) =
H n 2E
ms H msJ W W R W D W X
• LCMV (
2=): hard constraint
o speech source using free-field progation model:
o reference signal:
o solution:
( )
,1s s s
m
mX
mX d p
,1
s s
m m
D X
1 , 1
s
/
s H sn n
d d
W R R d
• Soft-constrained beamforming (
2):
o soft constraint on (partially) modelled speech distortion term:
model for spatial characteristics, on-line estimation of spectrum o speech source in region P:
o reference signal:
o solution:
( )
,1( )
s s s
m
X
m
p PX d p p
,1
( )
s s
m m
D X
p Pp
n
2P
Xs1 s( )
s H,( )
1
2P
Xs1 s( )
p Pd p d p
p Pd p
W R
on-line a-priori
Multi-microphone noise reduction
General cost function
- a-priori model - on-line model - combined model
Conclusions
99
• reference signal = microphone signal
• typically requires VAD + noise more stationary than speech
performance affected by VAD errors and non-stationary noise
On-line speech model (
On-line speech model (
22=0 =0 ) )
2
( ) =
H n 1E D
s H sJ W W R W W X
1
s s
D X
Multi-microphone noise reduction
General cost function
- a-priori model - on-line model - combined model
Conclusions
• TF-LCMV (
1=): hard constraint
o output speech component = speech component microphone signal
o on-line estimate of TF-ratio using non-stationarity of speech o adaptive implementation: TF-GSC, adaptive blocking matrix
,
1 1
1
s/
s HH s
n n
s
H
W H W R H R H
[Gannot 2001]
1010
• reference signal = microphone signal
• typically requires VAD + noise more stationary than speech
performance affected by VAD errors and non-stationary noise
• SDW-MWF (
1):
o soft constraint on speech distortion
o on-line estimate of speech correlation matrix using VAD
o SDW-MWF = TF-LCMV + single-channel postfilter spectral filtering
On-line speech model (
On-line speech model (
22=0 =0 ) )
2
( ) =
H n 1E D
s H sJ W W R W W X
1
s s
D X
n
1 s
1
1 1s
W R R r
Multi-microphone noise reduction
General cost function
- a-priori model - on-line model - combined model
Conclusions
11
• Combination of a-priori knowledge and on-line estimation
increase robustness against estimation errors
Combined speech model Combined speech model
2
1
2
( ) =
H n s H s 2 ms H smJ W W R W E D W X E D W X
• SDR-GSC (
2=)
o combination of LCMV beamformer and SDW-MWF
o hard constraint imposed through GSC-structure (FB + BM)
o soft constraint: on-line estimated speech distortion between speech component in speech reference and output signal
[Spriet 2004]
q a
W W B W
s H s 2
aH H s s H, a
E D W X E W B X X B W
Multi-microphone noise reduction
General cost function
- a-priori model - on-line model - combined model
Conclusions
• Soft-constrained SDW-MWF (
2):
o combination of SDW-MWF and soft-constrained beamformer o speech model:
– partially updated based on incoming data
– partially computed a-priori using model or calibration data
1212
• General cost function:
o output noise energy + speech distortion o on-line estimated vs. a-priori model
o speech distortion: hard vs. soft constraint
• Derivation of signal-dependent multi-microphone noise reduction algorithms:
o LCMV, TF-LCMV, SDW-MWF, soft-constrained beamformer, SDR-GSC, soft-constrained SDW-MWF
• Extensions and combinations:
o a-priori noise model: fixed beamforming
o combination of on-line and a-priori noise model: e.g. sensitivity- constrained GSC
o several other possibilities possible!
• Combination of a-priori knowledge and on-line estimation of both speech and noise terms anticipated to enhance robustness
Conclusions Conclusions
Multi-microphone noise reduction
General cost function
Conclusions