Relationship between Granger non-causality and network graph of state-space
representations
Jozsa, Monika
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date: 2019
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Jozsa, M. (2019). Relationship between Granger non-causality and network graph of state-space representations. University of Groningen.
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
Chapter 2
Granger causality and Kalman representations
in block triangular form
Kalman representations, introduced in Section1.2, are LTI–SS representations where the noise process is the innovation process of the output process. The noise and also the state processes of these representations can be expressed by using the output process, based on projections in Hilbert spaces generated by random variables of the output process. This interpretation of the noise and state processes helps us to connect the network graph of Kalman representations to properties of the output process. In this chapter we consider Kalman representations with a specific network graph which has two nodes and one directed edge. We call Kalman representations with this network graph, Kalman representations in block triangular form. Then we relate the existence of a Kalman representations in block triangular form to proper-ties of their output processes as follows: Given a partition y “ ryT
1, yT2sTof a process
y, there exists a Kalman representation of y in block triangular form if and only if y1
does not Granger cause y2. Kalman representations in block triangular form have
several advantageous properties, e.g., they allow for distributed parameter and state estimation.
In (Granger, 1963), Granger causality has already been characterized by proper-ties of VAR models. Also, the papers (Caines and Chan, 1975;Caines, 1976;Gevers
and Anderson, 1982) relate Granger causality to network graphs of MA models.
Therefore, the results of Chapter2can be viewed as a counterpart of the results in the cited papers for LTI–SS representations. The results presented here are not the first results on Granger causality in terms of LTI–SS representations. In (Barnett and Seth, 2015; Solo, 2016) Granger causality was characterized by properties of LTI– SS representation using transfer function approach. Contrary to (Barnett and Seth, 2015;Solo, 2016), we give a characterization for Granger non-causality by construct-ing Kalman representations in block triangular form. This chapter together with Chapter3is based on the journal paper (Jozsa et al., 2018b) and conference abstract (Jozsa et al., 2016).
The structure of this chapter is as follows: First, we introduce Kalman repre-sentations in block triangular form. Then, we characterize their existence in terms
of Granger causality. This will be followed by the presentation of two realization algorithms that calculate Kalman representations in block triangular form. Finally, we provide an example to illustrate the results. The poofs of the statements can be found in Appendix2.A. If not stated otherwise, we assume throughout this chapter that y “ ryT
1, y2TsT is a ZMSIR process where yiP Rri, and rią 0for i “ 1, 2.
2.1
Kalman representation in block triangular form
In this section we introduce Kalman representations in block triangular form and discuss their properties. To begin with, we define this class of representations:Definition 2.1. A Kalman representation pA, K, C, I, e “ reT
1, eT2sT, yq, where ei P
Rri, i “ 1, 2, is called a Kalman representation in block triangular form, if A “„A11A12 0 A22 K “„K11K12 0 K22 C “„C11C12 0 C22 , (2.1)
where Aij P Rpiˆpj, Kij P Rpiˆrj, Cij P Rriˆpj and pi ě 0 for i, j “ 1, 2. If,
in addition, pA22, K22, C22, Ir2, e2qis a minimal Kalman representation of y2, then pA, K, C, I, e, yqis called a Kalman representation in causal block triangular form.
Remark 2.2. If p2 “ 0 in Definition2.1, then the block matrices A12, A22, K22, C12
and C22are absent, whereas if p1 “ 0, then A11, A12, K11, K12and C11 are absent.
Furthermore, if p2“ 0, then y2“ e2is a white noise process and if p1“ p2“ 0, then
y “ eis a white noise process. In both cases, block triangular form implies causal block triangular form. A minimal Kalman representation pA, K, C, I, eq of a white noise process y has zero dimension (A, K, C are absent) and it is the trivial equation y “ e.
If pA, K, C, I, e, yq is a minimal Kalman representation in causal block triangular form satisfying (2.1) then the dimensions of the block matrices Aij P Rpiˆpj, Kij P
Rpiˆrj, C
ij P Rriˆpj, i, j “ 1, 2 are uniquely determined by y “ ryT1, yT2sT. Indeed,
pA22, K22, C22, Ir2, e2qis a minimal Kalman representation of y2 thus p2 is the di-mension of a minimal LTI–SS representation of y2. Furthermore, p1“ p ´ p2, where
pis the dimension of a minimal LTI–SS representation of y. That is, the dimension of a minimal LTI–SS representation of y2 and y and the dimensions of y1 and y2
determine the dimensions of Aij, Kij, Cij, i, j “ 1, 2.
Kalman representations in block triangular form can be viewed as a cascade in-terconnection of two subsystems, see Figure2.1. More precisely, let pA, K, C, I, eq be a Kalman representation of y in block triangular form satisfying (2.1) and let
2.1. Kalman representation in block triangular form 37
Sc Sa
x2, e2
Figure 2.1:Network graph of a Kalman representation in block triangular form: Sc
is the coordinator (2.2), Sais the agent (2.3).
x “ rxT
1, xT2sT be its state process where xi P Rpi, i “ 1, 2. Then we can define the
dynamical systems Scand Sabelow.
Sc " x2pt ` 1q “ A22x2ptq ` K22e2ptq y2ptq “ C22x2ptq ` e2ptq (2.2) Sa # x1pt ` 1q “ř 2 i“1pA1ixiptq ` K1ieiptqq y1ptq “ř 2 i“1C1ixiptqq ` e2ptq (2.3) The subsystem Sc, which generates y2, will be called coordinator, and the subsystem
Sa, which generates y1, will be called agent. The coordinator sends its state x2and
noise e2to the agent while the agent does not send information to the coordinator.
Accordingly, the network graph of pA, K, C, I, e, yq is the two-node star graph with Scbeing the root node and Sabeing the leave.
Motivation for Kalman representations in causal block triangular form
If we considered an LTI–SS representation of y “ ryT
1, yT2sT without requiring it to
be a Kalman representation, then in general the subsystems S1with output y1and
S2with output y2could change information in both direction via the noise process.
The fact that we require the LTI–SS representation to be a Kalman representation in causal block triangular form implies that this cannot be the case: notice that the second noise component e2is the innovation process of y2which implies that e2and
x2depend only on the past and present values of y2. In contrast, the first noise and
state components e1and x1depend on the past and present values of both y1and
y2. This ensures that, indeed, a Kalman representation in causal block triangular
form means that there is no communication from the agent to the coordinator. Kalman representations in causal block triangular form guarantee the subsystem which corresponds to the coordinator to be minimal and thus it is unique up to isomor-phism (see Definition 1.11and Proposition 1.12). An advantage of minimality is that it implies observability, and hence the state x2can be estimated from y2. This
opens up the perspective of distributed estimation, and possibly, with the future inclusions of inputs, of distributed control. An example for not requiring the sub-system of the coordinator to be minimal: Let us consider a Kalman representation
pA, K, C, I, e, yqin block triangular form with state process x “ rxT
1, xT2sT, where
y1, y2, e1, e2, x1P R, x2P R2and the system matrices are given as below.
A “ » – 0.5 0 0 0 0.6 0 0 0 0.7 fi fl, K “ » – 1 1 0 1 0 1 fi fl, C “ „1 1 1 0 0 1 .
The coordinator is then the subsystem with system matrices A22“ „0.6 0 0 0.7 , K22“ „1 1 , C22“ “ 0 1‰ .
Since the coordinator is not minimal, this Kalman representation in block triangular form is not in causal block triangular form. What happens is that the coordinator contains dynamics which could also be made part of the agent. For this example, it is not enough to know y2to estimate the state of the coordinator, for that we have
to use the output y1of the agent. Imposing minimality on the coordinator avoids
such degenerate cases.
2.2
Characterization of Granger non-causality by Kalman
representations in block triangular form
In this section we show that the existence of a Kalman representation in causal block triangular form characterizes Granger non-causality.
The next definition is a particular case of the concept of causality between stochastic processes defined in (Granger, 1963), if the latter is applied to ZMSIR processes, and if, using the terminology of (Granger, 1963), there is no external process.
Definition 2.3(Granger non-causality). Consider a ZMSIR process y “ ryT 1, yT2sT.
We say that y1does not Granger cause y2if for all t, k P Z, k ě 0
Elry2pt ` kq | H y2
t´s “ Elry2pt ` kq | H y t´s.
Otherwise, we say that y1Granger causes y2.
Informally, y1does not Granger cause y2, if for all k ě 0, the best k-step linear
prediction of y2 based on the past values of y2is the same as that of based on the
past values of y. Note that Definition2.3is equivalent to the weakly feedback free property of processes, see (Caines, 1988, Definition 2.1, Chapter 10). The notion of
2.2. Characterization of Granger non-causality by Kalman representations in block
triangular form 39
strongly feedback free property is equivalent to that for all k ě 0
Elry2pt ` kq | Ht´y2s “ Elry2pt ` kq | Hyt´` Hyt1s, (2.4)
i.e., when the k-step linear prediction of y2based on the past values of y2is the same
as that of based on the past values of y and the present values of y1. In (Granger,
1963), causality was defined as in Definition2.3, however, to a subclass of ZMSIR processes where Definition2.3implies (2.4). In the rest of the thesis we do not deal with causality as in (2.4).
Remark 2.4(Related work). If y is coercive, Granger non-causality from y1to y2is
further equivalent to, (see (Barnett and Seth, 2015))
@k ě 0 : pCpA ´ KCqkKq21“ 0, (2.5)
where p.q21denotes the r2ˆr1left lower block of a matrix and pA, K, C, I, eq is a
min-imal Kalman representation of y. The system matrices of a minmin-imal Kalman repre-sentation pA, K, C, I, e, yq in block triangular form naturally satisfy (2.5). Moreover, they assume block triangular Wold decomposition, i.e., block triangular transfer ma-trix between the innovation process e and y, see (Hsiao, 1982;Caines, 1976;Barnett and Seth, 2015) and (Caines, 1988, Theorem 2.2, Chapter 10). Note that in the cited papers, the reader can find equivalent formulation of block triangular Wold decom-position, e.g., in terms of feedback free property of feedback systems. In contrast to the cited papers, Theorem2.5below covers the case when y is non-coercive and in its proof the LTI–SS representation pA, K, C, I, e, yq that characterizes Granger non-causality in y is constructed.
Below, we state the main result of this chapter :
Theorem 2.5. Consider the following statements for a ZMSIR process y “ ryT 1, yT2sT :
(i) y1does not Granger cause y2;
(ii) there exists a minimal Kalman representation of y in causal block triangular form; (iii) there exists a minimal Kalman representation of y in block triangular form; (iv) there exists a Kalman representation of y in block triangular form;
Then(i) ðñ (ii). If y is coercive, then(i) ðñ (ii) ðñ (iii) ðñ (iv).
Theproofcan be found in Appendix2.A. Intuitively, Granger non-causality in Theorem2.5means that for predicting y2, there is nothing to be gained from the
of y in causal block triangular form the subsystem pA22, K22, C22, Ir2, e2qof y2 de-pends only on e2 and thus only on the past values of y2. On the other hand, the
subsystem which generates y1depends on the entire history of y. Theorem2.5can
also be interpreted as follows: Granger non-causality from y1 to y2 is equivalent
to the process y “ ryT
1, yT2sT admitting a minimal Kalman representation with
net-work graph depicted on Figure2.1.
Theorem2.5relates Granger non-causality with existence of minimal Kalman rep-resentations in block triangular form. Since all minimal Kalman reprep-resentations of y are isomorphic, Theorem2.5not only guarantees that Granger non-causality trans-lates into existence of a Kalman representation with a suitable network graph, but also guarantees that any minimal Kalman representation of y is isomorphic to this particular one. Hence, most of the interesting dynamical properties of this Kalman representation are also valid for any other minimal Kalman representation. Further-more, any minimal Kalman representation can be brought to this specific one via a linear state-space transformation. Since black-box identification algorithms, for example subspace methods, yield minimal Kalman representations, Theorem2.5is also interesting for deriving and interpreting network graphs based on data.
2.3
Computing Kalman representation in block
trian-gular form
In this section we present a procedure for constructing a minimal Kalman represen-tation in causal block triangular form.
Consider an LTI–SS representation p ¯A, ¯B, ¯C, ¯D, vq of y “ ryT
1, yT2sT. Then it
can be transformed into a minimal Kalman representation p ˆA, ˆK, ˆC, I, eqof y using Algorithm2. Now take the partition ˆC “”Cˆ1T Cˆ2T
ıT
such that the number of rows of ˆCiequals ri“ dimpyiqfor i “ 1, 2. Furthermore, define a non-singular matrix T
such that pT ˆAT´1, ˆC
2T´1qis in the form (see, e.g., (Rosenbrock, 1970))
T ˆAT´1 “„A11A12 0 A22 , ˆC2T´1“ “ 0 C22‰ , (2.6)
where pA22, C22qis observable and A11 P Rp1ˆp1, A22 P Rp2ˆp2 such that p2 is the
rank of the observability matrix of the pair p ˆC2, ˆAq. Note that if p ˆA, ˆC2qis observable,
then p1“ 0and A11, A12are absent in (2.6). In addition, if the observability matrix
2.3. Computing Kalman representation in block triangular form 41 (2.6). Note that such a matrix T always exists, (Rosenbrock, 1970). Define
A :“ T ˆAT´1, K :“ T ˆK, C :“ ˆCT´1 (2.7)
and consider the partition A “„A11A12 A21A22 , K “„K11K12 K21K22 , C “„C11C12 C21C22 , (2.8)
where Aij P Rpiˆpi, Kij P Rpiˆrj, Cij P Rriˆpi, i, j “ 1, 2 and from (2.6), A21 “
0, C21“ 0. Based on Theorem2.5we can state the following result:
Corollary 2.6. The following statements hold:
• If y1does not Granger cause y2, then either K21is absent or K21“ 0. Furthermore,
pA, K, C, I, eqis a minimal Kalman representation of y “ ryT
1, yT2sT in causal block
triangular form with Aij, Kij, Cij, i, j “ 1, 2 defined by (2.6), (2.7) and (2.8).
• If y is coercive, then the absence of K21or K21“ 0implies that y1does not Granger
cause y2.
Theproofcan be found in Appendix2.A. Corollary2.6yields a method to calcu-late a minimal Kalman representation in causal block triangular form in the absence of Granger causality. This idea is elaborated in Algorithms 4 and 5 that rely on Algorithms1and 2. Algorithm 4takes an LTI–SS representation as its input and transforms it into a minimal Kalman representation in causal block triangular form. Algorithm5calculates the same representation from output covariances. Hence, by using empirical covariances, it can be applied to data. Note that in Algorithms4–5
the dimensions ri“ dimpyiq, i “ 1, 2 are predefined.
Algorithm 4Minimal Kalman representation in causal block triangular form based on LTI–SS representation
Input t ¯A, ¯B, ¯C, ¯D, Λv
0u: System matrices of an LTI–SS representation
p ¯A, ¯B, ¯C, ¯D, vqof y and variance of v
Output tA, K, Cu: System matrices of (2.1)
Step 1Apply Algorithm 2with input t ¯A, ¯B, ¯C, ¯D, Λv
0uand denote its output by
t ˆA, ˆK, ˆCu.
Step 2Let ˆC ““ ˆCT 1Cˆ2T
‰T
be such that ˆCiP Rriˆn. Calculate a non-singular matrix
Tsuch that (2.6) holds and pA22, C22qis observable.
Algorithm 5Minimal Kalman representation in causal block triangular form based on output covariances
Input tΛyku 2N
k“0: Covariance sequence of y ““y T 1, y
T 2
‰T
Output tA, K, Cu: System matrices of (2.1)
Step 1Apply Algorithm1with input tΛyku 2N
k“0and denote its output by t ˆA, ˆK, ˆCu.
Step 2Steps 2–3 of Algorithm4.
Remark 2.7 (Correctness of Algorithms 4–5). Consider a ZMSIR process y “ ryT1, y2TsT with covariance sequence tΛ
y
ku8k“0and an LTI–SS representation p ¯A, ¯B, ¯C,
¯
D, vqof y. Let e be the innovation process of y and N be any number larger than or equal to the dimension of a minimal LTI–SS representation of y. Assume that y satisfies condition(i)of Theorem2.5and note that Algorithms1–2calculate a mini-mal Kalman representation (Remark1.8). Then from it follows Corollary2.6that if tA, K, Cuis the output of Algorithm4with input t ¯A, ¯B, ¯C, ¯D, Λv
0 “ ErvptqvTptqsu,
then pA, K, C, I, eq is a minimal Kalman representation in causal block triangular form. Similarly, it follows from Corollary2.6that if tA, K, Cu is the output of Algo-rithm5with input tΛu2N
k“0, then pA, K, C, I, eq is a minimal Kalman representation
of y in causal block triangular form.
Remark 2.8. In a similar fashion as in Remark1.9, Algorithms5and4have poly-nomial complexity. Algorithm4is polynomial in the dimensions of the state, out-put and noise processes of the LTI–SS representation p ¯A, ¯B, ¯C, ¯D, vq. Algorithm5is polynomial in the number and size of the output covariances.
Remark 2.9(Checking Granger non-causality). Algorithms4and5can be used to check Granger non-causality by looking whether the left lower block of matrix K is zero (K21 “ 0) in the partition (2.8), where tA, K, Cu are the matrices returned
by Algorithm4 or5. If K21 ‰ 0, then y1 Granger causes y2. If y2is coercive and
K21is absent or K21 “ 0, then y1does not Granger cause y2in the view of
Corol-lary2.6. If y is non-coercive, then it should be checked if pA22, K22, C22, Ir2, e2qis a minimal Kalman representation of y2. This can be done by computing a
mini-mal Kalman representation p ˜A22, ˜K22, ˜C22, I, ˜e2qof y2using Algorithm1or2. If the
noise variance Ere2ptqeT2ptqsis equal to the new noise variance Er˜e2ptq˜eT2ptqs, then
from (Dufour and Renault, 1998, Proposition 2.3) we know that y1does not Granger
2.4. Example for block triangular representation 43
2.4
Example for block triangular representation
In this section we adopt a simplified version of a case study in (Kempker, 2012, Section 8.1) to illustrate the results of this chapter. The focus of this study is the dynamics of underwater vehicles that track a reference path in a fixed formation. Among the vehicles there is one acting as a coordinator that tracks a reference path and the others acting as agents that track the coordinator. For our purpose, we consider two underwater vehicles that track a reference path in a fixed distance, where one of them is the coordinator and the other is an agent.
In comparison with (Kempker, 2012, Section 8.1), besides the number of vehicles, we made the following changes:
• to ensure stationarity, the coordinator follows the zero position
• for convenience, we consider the movements of the vehicles along the first coordinate
• besides the position disturbance we include measurement noise.
We will show that the relative positions (concerning the fixed distance) of the vehicles are ZMSIR processes that can be modeled by a minimal Kalman representa-tion in causal block triangular form. In fact, we reverse engineer the network graph from the observed process in the following way: Denote the agent vehicle by y1and
the coordinator vehicle by y2. Then we verify that y1does not Granger cause y2.
Based on Remark2.7, this allows us to calculate a minimal Kalman representation in causal block triangular form using Algorithm5.
Model description: Assume that we have two underwater vehicles V1 and Vc
where V1 acts as an agent and Vc as the coordinator. For j P t1, cu denote the first
coordinate at time t P Z of the position, velocity, acceleration, position disturbance and measurement noise of Vj by pjptq, sjptq, ajptq, wjptq and ˜wjptq, respectively.
Also, denote the first coordinate of the reference position and velocity of Vjby pRjptq
and sR
jptq, respectively. Let
pRcptq “ ´ppcptq ` ˜wcptqq
pR1ptq “ ppcptq ` ˜wcptqq ` ∆1.
That is, Vc follows the zero position based on its own measured position and V1
follows Vcin a distance ∆1based on the same information.
The dynamics of rpj, sjsT, j P t1, cu is given by
„pjpt ` 1q sjpt ` 1q “„1 1 0 τ ´1 τ „pjptq sjptq `„ 01 τ aj` „1 0 wjptq (2.9)
where ajis the control input and τ is a time constant. The reference signals rpRj, s R js
T
are estimated by an observer with dynamics „ ˆpR jpt ` 1q ˆ sR jpt ` 1q “„1 ´ G p j 1 ´Gsj τ ´1τ „ ˆpR jptq ˆ sR jptq `„G p j Gs j pRjptq, (2.10)
where Gpj, Gsjare constant gains. The linear feedback control is then given by
aj “ ” FjpFjs ı„pj´ ˆpR j sj´ ˆsRj .
Combining (2.9) and (2.10) we obtain the closed loop system » — — – pjpt`1q sjpt`1q ˆ pRjpt`1q ˆ sR jpt`1q fi ffi ffi fl “ » — — — – 1 1 0 0 1 τF p j τ ´1τ ` 1 τF s j ´1τ F p j ´1τ F s j 0 0 1´Gpj 1 0 0 ´Gsj τ ´1τ fi ffi ffi ffi fl loooooooooooooooooooomoooooooooooooooooooon Aj » — — – pjptq sjptq ˆ pRjptq ˆ sR jptq fi ffi ffi fl ` » — — – 0 0 Gpj Gs j fi ffi ffi fl loomoon Bj pRjptq ` » — — – 1 0 0 0 fi ffi ffi fl loomoon E wjptq.
Note that x1 :“ rp1´ ∆1, s1, ˆpR1 ´ ∆1, ˆsR1sT has essentially the same dynamics as
rp1, s1, ˆpR1, ˆs R 1s
T, namely
x1pt ` 1q “ A1x1ptq ` B1pRc ` Ew1ptq.
Assuming that v :“ rw1, wc, ˜w1, ˜wcsT is a white noise process we can define the
following LTI–SS representation of the process y “ ry1, ycsT :“ rp1´ ∆1, pcsT:
„x1pt ` 1q xcpt ` 1q “„ A1 “ B1 0 ‰ 0 Ac´BcF loooooooomoooooooon A „x1ptq xcptq `„ E 0 0 B1 0 E 0 ´Bc looooooomooooooon B vptq „y1ptq ycptq “„ E T 0 0 ET looooomooooon C „x1ptq xcptq `„ 0 1 0 0 0 0 0 1 loooomoooon D vptq.
Parameter settings: Following the approach of (Kempker, 2012), we take F1p“ F p c
and Fs
1 “ Fcsas the solution of the linear quadratic problem minact||z
2
1||2` α||a2c||2u
with respect to the dynamics „z1pt ` 1q z2pt ` 1q “„1 1 0 τ ´1τ „z1ptq z2ptq `„ 01 τ acptq.
2.4. Example for block triangular representation 45 Accordingly, for τ “ 2 and α “ 10 the optimal solution is F1p“ F
p
c “ ´0.3and F1s“
Fs
c “ ´0.5. The gain constants were chosen to be G p
1 “ 1.5, Gs1 “ 0.3, G p
Rc “ 0.9, and Gs
Rc “ 0.5for which the matrix A is stable. Finally, the joint noise process v is chosen to be a normalized Gaussian white noise process.
Reverse engineering of the network graph: Assume that the output process y of the LTI–SS representation S :“ pA, B, C, D, v, yq is observed. Using the result of this chapter, we will calculate a minimal Kalman representation of y in causal block triangular form. Note that we do not use prior knowledge of the structure of the network graph. In fact, this representation reconstructs the network graph of S.
First, we check the Granger non-causal relations among the components of y using the covariance sequence tΛyku
2N
k“0 of y where N is larger than or equal to
the dimension of a minimal LTI–SS representation of y. For this, we calculate a Kalman representation pA, K, C, I, eq of y and verify that y is coercive by checking that A ´ KC is invertible (see Section1.2.2). In view of Corollary2.6, a Granger non-causal relation can be verified by observing the output matrix K of Algorithm5. More specifically, if the left lower block of the matrix K is zero, then an appro-priate Granger non-causal relation holds (see Remark2.9). Following this method, we apply Algorithm5choosing the coordinator to be y1 and yc, respectively. We
obtain that y1 does not Granger cause yc the condition of Theorem 2.5holds for
y “ ry1, ycsT.
Second, in order to calculate a Kalman representation of y in block triangular form, we apply Algorithm 5 with the covariance sequence tΛyku
2N
k“0 as its input.
With our parameter settings, the Kalman representation, that the output matrices of Algorithm5defines, is given as follows:
« x1pt ` 1q xcpt ` 1q ff “ » — — — — — — — — — — — — – 0.4 ´0.3 ´0.1 0.1 0.0 0.1 0.2 ´0.1 0.2 0.4 0.6 0.2 0.1 0.0 0.2 0.1 0.0 ´0.3 0.4 ´0.1 0.0 0.0 0.0 ´0.4 ´0.2 0 ´0.2 0.7 0.0 ´0.1 0.0 0.0 0 0 0 0 0.2 ´0.9 0.1 0.0 0 0 0 0 0.6 0.3 0.3 0.1 0 0 0 0 ´0.1 0.2 ´0.4 0.4 0 0 0 0 0.1 0.0 ´0.5 0.3 fi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi fl loooooooooooooooooooooooooooooooomoooooooooooooooooooooooooooooooon Ak « x1ptq xcptq ff ` » — — — — — — — — — — — — – 0.1 0.2 ´0.1 ´0.1 0.0 0.0 ´0.3 ´0.1 0 0.1 0 0.2 0 0.2 0 ´0.1 fi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi fl loooooooomoooooooon K e1 « y1ptq ycptq ff “ « ´0.4 ´0.3 0.4 ´1.8 0.2 ´0.2 0.0 0.2 0 0 0 0 0.5 0.0 ´0.2 0.0 ff loooooooooooooooooooooooooooomoooooooooooooooooooooooooooon Ck « x1ptq xcptq ff ` e.
representation in causal block triangular form. The calculation of Skonly requires
the second order statistics of the output process and does not use prior knowledge of the network topology. Therefore, the construction of Sk shows that the network
graph of S can be reverse engineered. Moreover, by Theorem3.5, the reconstructed representation Sknot only has block triangular structure, but also characterizes the
causal relations in the observed process. The procedure can be repeated based on data, using empirical covariances which provide an estimation of Sk. Note that
Skcould also be calculated in a distributed way which possibly reduces estimation
error when applied to data.
2.5
Conclusions
In this chapter we characterized Granger non-causality between two components of a stationary process with the existence of a minimal Kalman representation in a so-called causal block triangular form that generates the process at hand. This rep-resentation not only characterizes Granger non-causality but has a network graph that illustrates the causal relationship between the components of its output process. We provided algorithms for calculating this Kalman representation, in particular, calculating it from the covariance sequence of the observed output process. The co-variances can be estimated from data. Hence, our results open up the possibility of calculating this representation from output data. The results deal with coercive and non-coercive processes and the minimality of the representations.
2.A. Proof of Theorem2.5and Corollary2.6 47
2.A
Proof of Theorem
2.5
and Corollary
2.6
Recall that U `V :“ tu`v|u P U, v P V u denotes the sum of two subspaces U, V Ď H, V a Udenotes the orthogonal complement of U in V , U 9`V denotes the direct sum and U ‘ V denotes the orthogonal direct sum of U in V . Furthermore, an LTI-SS representation pA, B, C, D, vq of a white noise process y has zero dimension, thus A, B, Care absent, and it is the trivial equation y “ Dv. A zero dimensional LTI-SS representation is minimal, observable and controllable by convention.
Proof of Theorem2.5. First, we discuss the trivial implications: since any minimal Kalman representation in causal block triangular form is a Kalman representation in a causal block triangular form and any Kalman representation in a causal block triangular form is a Kalman representation in block triangular form (ii) ùñ (iii)
and(iii) ùñ (iv)follow. In addition, the implication(ii) ùñ (i) is easy to see; if pA, K, C, I, eqis a minimal Kalman representation of y in causal block triangular form (2.1), then pA22, K22, C22, Ir2, e2qis a minimal Kalman representation of y2, and hence e2ptq “ y2ptq ´ Elry2ptq | H
y
t´s equals the innovation process of y2.
By (Dufour and Renault, 1998, Proposition 2.3), the latter implies that y1 does not
Granger cause y2.
(iv) ùñ (i)if y is coercive: Since y is coercive, Granger non-causality is equiv-alent to the transfer matrix of a Kalman representation of y having a block trian-gular structure, see (Caines, 1976, Theorem 2.2.). Since the transfer function of a Kalman representation in block triangular form has a triangular structure described in (Hsiao, 1982;Caines, 1976), the implication(iv)ùñ(i)follows.
(i)ùñ(ii): To begin with, from Proposition1.7we know that the ZMSIR process y “ ryT
1, yT2sT has a minimal Kalman representation p ˆA, ˆK, ˆC, I, eq. Assuming that
(i)in Theorem2.5holds, we transform this representation into causal block triangu-lar form. Consider the partition ˆC “ r ˆCT
1, ˆC2TsT such that Ci P Rriˆpwhere ri “
dimpyiqis the dimension of yi, i “ 1, 2 and p is the dimension of p ˆA, ˆK, ˆC, I, e, yq.
We assume that p ą 0; if p “ 0, then y “ e defines a minimal Kalman representa-tion in causal block triangular form. Take the non-singular matrix T which brings p ˆA, ˆC2qinto observability staircase form, i.e., T is such that
T ˆAT´1 “„A11 A12 0 A22 , ˆC2T´1“ “ 0 C22‰ , (2.11)
where pA22, C22qis observable and A11 P Rp1ˆp1, A22 P Rp2ˆp2 such that p2 is the
rank of the observability matrix of the pair p ˆA, ˆC2q. Define A :“ T ˆAT´1, K :“
T ˆK, C :“ ˆCT´1and notice that pA, K, C, I, e, yq is a minimal Kalman representation
p1 “ 0and A11, A12are absent in (2.11). If the observability matrix of p ˆA, ˆC2qhas
zero rank, then p2 “ 0 and A12, A22, C22 are absent. Moreover, if p2 “ 0, then
pA, K, C, I, e, yqis already in causal block triangular form (see Remark2.2). Hence, we can assume that p2ą 0. Next, we show that K21“ 0where
K “„K11 K12 K21 K22
, Kij P Rpiˆrj for i “ 1, 2.
Denote the state of pA, K, C, I, e, yq by x. Take the partition e “ reT
1, eT2sT and x “
rxT1, xT2sT where eiP Rriand xi P Rpi, i “ 1, 2. Notice that
C “„C11 C12 0 C22 , Ak“„A k 11 pAkq12 0 Ak22 ,
where C1i P Rr1ˆpi, i “ 1, 2 and pAkq12 P Rp1ˆp2 denotes the right upper block of
Ak. It then follows that C
2Akxptq “ C22Ak22x2ptqand for k ą 0 y2pt ` kq “ C22Ak22x2ptq ` k´1 ÿ l“0 Mlept ` k ´ lq (2.12)
for some matrices M0, . . . , Mk´1. Since e is the innovation process of y, it implies
that ept`k´lq is orthogonal to Hyt´and Ht´e “ H y
t´, for k´l ě 0, t P Z. Furthermore,
from xptq “ř8
k“1CAk´1Kept ´ kq, the components of xptq belong to H e t´ “ H
y t´.
Using (2.12), it then follows that Elry2pt ` kq|Hyt´s “ C22Ak22x2ptq. As y1does not
Granger cause y2 we know that Elry2pt ` kq|Ht´y2s “ Elry2pt ` kq|Ht´y sfor all k ě
0, and thus Elry2pt ` kq|H y2
t´s “ C22Ak22x2ptq P H y2
t´. Let O2 be the observability
matrix of pA22, C22qand denote its left inverse by O`2. Then by defining Y2ptq “
ryT2ptq, ¨ ¨ ¨ , yT2pt ` n ´ 1qsT, we have that x2ptq “ O`2ElrY2ptq|Hyt´2sand thus the
elements of x2ptqbelong to H y2
t´. Note that since y1does not Granger cause y2, by
(Dufour and Renault, 1998, Proposition 2.3) e2is the innovation process of y2, and
hence Hy2 pt`1q´“ H y2 t´‘ H e2 t . Therefore, x2pt ` 1q “ Elrx2pt ` 1q|H y2 pt`1q´s “ Elrx2pt ` 1q|H y2 t´s ` Elrx2pt ` 1q|Het2s.
From that eptq is orthogonal to Hyt´Ě Elrx2pt ` 1q|H y2
t´sand Elrx2pt ` 1q|Het2s Ď Het
we have that
Elrx2pt ` 1q|Hets “ Elrx2pt ` 1q|Het2s “ ˆRe2ptq
for a suitable ˆRmatrix. Then x2pt ` 1q “ A22x2ptq `
“
2.A. Proof of Theorem2.5and Corollary2.6 49 orthogonal to Hx2 t implies that Elrx2pt ` 1q|Hets “ “ K21 K22‰ eptq “ ˆRe2ptq.
Using that y is full rank, e1and e2 are linearly independent, and hence K21 “ 0,
K22“ ˆR. That is, pA, K, C, I, eq is a Kalman representation of y in block triangular
form.
In order to see that pA, K, C, I, eq is in causal block triangular form, we need to show that pA22, K22, C22, Ir2, e2qis a minimal Kalman representation of y2. From Granger non-causality, e2 is the innovation process of y2, hence we only need to
prove minimality. Note that if p1 “ 0, then A “ A22, K “ K22, C “ C22 thus
pA22, K22, C22, Ir2, e2qis minimal. In view of Proposition1.10, it is sufficient to show that pA22, C22qis observable and pA22, K22qis controllable. The former follows from
the construction. Assume now indirectly that pA22, K22qis uncontrollable, i.e., that
for some vector η ‰ 0, ηTAk
22K22“ 0for all k ě 0. However,
AkK “„A k 11K11 pAkKq12 0 Ak 22K22 , where pAkKq
12denotes the right upper block of AkKwith suitable dimensions. It
follows that“0 ηT‰ Ak
K “ 0for all k ě 0, which implies that pA, Kq is not control-lable. Since pA, K, C, I, eq is a minimal Kalman representation of y, by Proposition
1.10pA, Kqis controllable, which is a contradiction. This implies that pA, K, C, I, eq is a minimal Kalman representation in causal block triangular form which completes the proof.
Proof of Corollary2.6. The construction of the Kalman representation pA, K, C, I, eq of y coincides with the one described in the proof of the implication(i) ùñ (ii)in Theorem2.5. Hence, if y1 does not Granger cause y2, then the above-mentioned
proof implies that either K21is absent or K21 “ 0and that pA, K, C, I, eq is a
mini-mal Kalman representation of y in causal block triangular form. Conversely, if K21
is absent or K21 “ 0, and y is coercive, then pA, K, C, I, eq is a minimal Kalman
representation of y in block triangular form. Hence, by the implication(iii)ùñ (i)
