Supervisors: AND APPLICATIONS IN TENSORIZATION

TENSORIZATION

AND APPLICATIONS IN

BLIND SIGNAL SEPARATION

OTTO DEBALS

Supervisors:

Prof. Lieven De Lathauwer

Prof. Marc Van Barel

Data

Patterns/

relations

Predictions

Self-driving car algorithms

Predictions:

behaviors

Traffic & congestion models

Predictions:

time to goal

Electrocardiogram analysis

Predictions:

disorders

Data

Patterns/

relations

Predictions

Data representation?

Mining tools?

VECTOR = ONEWAY ARRAY

22 24 29 23 25 25 26

5.1 2.7 9.4 8.8 1.7 2.2 3.4

[

]

Mon Tue Wed Thu Fri Sat Sun

R0 E40 R1 E19 E17 E314 R4

Temperature in °C

MATRIX = TWOWAY ARRAY

Mon Tue Wed Thu Fri Sat Sun

MATRIX = TWOWAY ARRAY

Mon Tue Wed Thu Fri Sat Sun

MATRIX = TWOWAY ARRAY

Mon Tue Wed Thu Fri Sat Sun

Leuven

22 24 29 23 25 25 26

21

23 22 21

23 23 22

27 29 28 26 26 27 26

30 30 31

32 32 31

32

Day

Week

Day

TENSOR = MULTIWAY ARRAY

Sheet 1

Sheet 2

Sheet 3

IN TERMS OF

DATA MINING TOOLS

Matrix tools

Information

Information

Tensor tools

VERY POWERFUL!

WHAT EXACTLY IS

Data

Application of

_{tensor tools}

Data

Application of

_{tensor tools}

Naturally

Data

Application of

_{tensor tools}

Naturally

Physical quantities in function of height x width x depth

Video data: height x width x time

Color image data, hyperspectral data, …

Data

Application of

_{tensor tools}

Naturally

By experiment design

Recommendation systems: user x movie → user x movie

x month x topic x …

Biomedical signal processing: electrode x time → electrode x time

x subject x …

Face databases: height x width → height x width

x person x emoticon x …

Data

Application of

_{tensor tools}

Naturally

By experiment design

&

Data

Application of

_{tensor tools}

Tensorization

Data

Application of

_{tensor tools}

Tensorization

&

Without using additional data

In a meaningful way

MORE ADVANCED

EXAMPLE: HANKELIZATION

a b c d e

a

b

c

b

c

d

c

d

e

MORE ADVANCED

EXAMPLE: HANKELIZATION

a b c d e

a

b

c

b

c

d

c

d

e

MORE ADVANCED

EXAMPLE: HANKELIZATION

a b c d e

a

b

c

b

c

d

c

d

e

VARIOUS TENSORIZATION

TECHNIQUES EXIST

Tensorization

overview

Single

vector

vectors

Sets of

Hankelization Segmentation Löwnerization Monomial relations Time-frequency & time-scale Moments & cumulants Adjacency tensors Score functions Hessian & Jacobian matrices Covariance matrices Piecewise outer product

And many

more

BLIND SIGNAL

Source signals

Mixing level

Observed signals

Source signals

Mixing level

Observed signals

Source signals

Mixing level

Observed signals

Assumption of

independence

allows the

recovery of

underlying components

Emission spectra of individual analytes

Observed emission spectra of the compound at different

excitation levels Influence of

excitation and concentrations

Emission spectra of individual analytes

Observed emission spectra of the compound at different

excitation levels Influence of

excitation and concentrations

Emission spectra of individual analytes

Observed emission spectra of the compound at different

excitation levels Influence of

excitation and concentrations

Assumption of

rational functions

allows

the recovery of

underlying components

Maternal and fetal

ECG signals Mixing level Observed ECG signals

Maternal and fetal

ECG signals Mixing level Observed ECG signals

Maternal and fetal

ECG signals Mixing level Observed ECG signals

Maternal and fetal

ECG signals Mixing level Observed ECG signals

Maternal and fetal

ECG signals Mixing level Observed ECG signals

Assumption of

independence / rational functions /

sums-of-Kronecker-products

allows the recovery of

underlying components

Maternal and fetal

ECG signals Mixing level Observed ECG signals

Assumption of

independence / rational functions /

sums-of-Kronecker-products

allows the recovery of

underlying components

HOW CAN WE USE

TENSORS

TO SOLVE THE PROBLEM OF

Translate assumptions

TENSORIZATION

TWO THINGS ARE NEEDED.

1

Different assumptions require different tensorization techniques: Independence

Rational functions Etc.

STATISTICS

LÖWNERIZATION Exponential polynomials HANKELIZATION

Translate assumptions

Identify underlying

components

TENSORIZATION

TENSOR TOOLS

TWO THINGS ARE NEEDED.

1

2

Different assumptions require different tensorization techniques: Independence

Rational functions Etc.

STATISTICS

LÖWNERIZATION Exponential polynomials HANKELIZATION

R0 8h

80

40

120

20

R0 E40 8h

80

40

120

20

20

10

30

5

R0 E40 R1 8h

80

40

120

20

20

10

30

5

40

20

60

10

R0 E40 R1 ChurchStr. 8h

80

40

120

20

20

10

30

5

40

20

60

10

4

2

6

1

RANK-1 MATRIX

R0 E40 R1 ChurchStr. 8h

80

40

120

20

20

10

30

5

40

20

60

10

4

2

6

1

RANK-1 MATRIX

R0 E40 R1 ChurchStr. 8h

80

40

120

20

20

10

30

5

40

20

60

10

4

2

6

1

x20

RANK-1 MATRIX

R0 E40 R1 ChurchStr. 8h

80

40

120

20

20

10

30

5

40

20

60

10

4

2

6

1

=

4

2

6

1

20 5

10

1

RANK-1 MATRIX

R0 E40 R1 ChurchStr. 8h

80

40

120

20

20

10

30

5

40

20

60

10

4

2

6

1

=

4

2

6

1

20

5

10

1

RANK-1 MATRIX

60 = 6 x 10

R0 E40 R1 ChurchStr. 8h

80

40

120

20

20

10

30

5

40

20

60

10

4

2

6

1

=

4

2

6

1

20 5

10

1

=

Normal behavior

=

+

…

+

=

+

…

+

=

+

…

+

=

+

…

+

=

=

+

…

+

=

+

…

+

=

+

…

+

=

+

…

+

Normal

behavior major incidentInfluence of

Normal behavior and

=

+

…

+

=

+

…

+

Normal

behavior major incidentInfluence of Normal behavior and

TENSOR DECOMPOSITIONS ALLOW

THE

UNIQUE RECOVERY

OF

TENSOR-BASED BLIND SIGNAL SEPARATION

TENSOR-BASED BLIND SIGNAL SEPARATION

+ =

1

TENSOR-BASED BLIND SIGNAL SEPARATION

+ =

1

COCKTAIL PARTY PROBLEM

TENSORIZATION

ONLY USEFUL

FOR SIGNAL SEPARATION?

AS A PARADIGM, FOR MUCH MORE!

Graph or data clustering Training neural networks Function approximations

Decoupling of systems Filter banks