On model parametrization and model structure selection for identification of MIMO-systems

(1)

On model parametrization and model structure selection for

identification of MIMO-systems

Citation for published version (APA):

Janssen, P. H. M. (1988). On model parametrization and model structure selection for identification of MIMO-systems. Technische Universiteit Eindhoven. https://doi.org/10.6100/IR284813

DOI:

10.6100/IR284813

Document status and date: Published: 01/01/1988

Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

(2)

On Model Parametrization and

Model Structure Selection

for Identification of

MIMO-Systems

Proefschrift

Ter verkrijging van de graad van doctor aan de Technische Universiteit Eindhoven, op gezag van

de rector magnificus, prof. dr. F.N. Hooge, voor een commissie aangewezen door het college van

decanen in het openbaar te verdedigen op dinsdag 31 mei 1988 te 16.00 uur.

door

Peter Hubertus Maria Janssen

(3)

door de prornotoren

prof. dr. ir.

P. Eykhoff

en

prof. ir. O. H; Bosgra

Coprornotor dr. ir. A. A. H. Darnen

CIP-GEGEVENS KONINKLIJKE BIBLIOTHEEK, DEN HAAG

Janssen, Peter Hubertus Maria

On model parametrization and model structure selection for

identification of MIMe-systems / Peter Hubertus Maria Janssen.

[S.l. : s.n.]. Fig., tab.

Proefschrift Eindhoven. - Met lit. opg., reg.

ISBN 90-9002221-X

SISO 656.2 UDC 519.71.001.3(043.3) NUGI832

(4)

'Wir wissen zwar nicht was wir wollen aber das mit voller Kraft'

Anonymus

'It is impossible to achieve the aim without suffering' J.G. Bennett on "Exposure" by Robert Fripp

voor mijn ouders aan R.

(5)

Een luie stoel, een lekker drankje, een prettig muziekje (heel tevree met J.S.B.)! Biermee lijkt de ideale entourage geschetst te zijn voor een van de laatste activiteiten bij het schrijven van een proefschrift, narnelijk het produceren van een voorwoord c.q. dankwoord.

Belaas, dit alles is slechts schijn!!! Het lustprincipe behoort nog tot het dome in van de dromen en fantasieen terwijl het lastprincipe nog steeds heer en meester is over de werkelijkheid. Zoals veelal gebruikelijk is bij het afronden van een proefschrift, is de race tegen de klok nog in volle gang en krijg je geen tijd om ontspannen en mijmerend terug te blikken. Snel ornkijken en blijven rennen, zonder te struikelen, luidt het devies. Geen ideale omstandigheden om stil te staan bij de mensen die de voorbije tijd nauw betrokken zijn geweest bij het werk. Toch maar een poging wagen!

In de eerste plaats wil ik vooral Ad Darnen, Petre Stoica en Paul Van den Hof bedanken.

Aan Ad heb ik heel veel steun gehad hij het schrijven van deze dissertatie. Ala co-promotor heeft hij uiterat nauwgezet de vele bladzijden papier van de diverse concept-versies doorgelezen en van veel nuttig commentaar voorzien. Gezien de omvang (en de inhoud) van het proefschrift een heel zware opgave, een marteling welhaast. Het heeft me dan ook telkens verbaasd dat hij, ondanks de vele bladzijden papier die ik tot vervelens toe op zijn bureau deponeerde, zijn commentaar steeds op een zeer positieve manier aan mij overhracht. Veel bewondering en waardering heb ik ook voor de originele en verfrissende ideeen en inzichten die hij daarbij aandroeg. Next I like to thank dr. Petre Stoica from the Institutul Polytechnic Bucuresti, who was the great inspirator behind Chapter 6 of this thesis. Despite of his overfull schedule, he was always willing to give very detailed and useful comments on my correspondence. His clever ideas and suggestions have led to new insights and have greatly improved the original versions of this materiaL With pleasure I think back to his visit of the Eindhoven University of Technology, and I regret that he was not able to be a member of the "Promotie Commissie" due to his teaching duties. Met heel veel plezier denk ik ook terug aan de tijd dat Paul Van den Hof nog aan de Technische Universiteit Eindhoven werkte. De samenwerking met hem was heel inspirerend, en van zijn inzichten heb ik vee I geleerd. De weerslag van discussies met hem en van zijn ideeen is dan ook op diverse plaatsen in dit proefschrift terug te vinden. Hoewel onze samenwerking minder hecht werd door zijn vertrek naar de Technische Universiteit Delft, en door het feit dat ieder van ons zijn eigen koers moest varen bij het voltooien van het proefschrift, deed ik nooit tevergeefs een beroep op hem. Toen ik bijv. in uiterste tijdnood verkeerde bij de afronding van de dissertatie, is hij bijgesprongen en heeft de figuren uit Sectie 4.5 aangeleverd.

(6)

ii

Vervolgens wi! ik de beide promotoren prof. Pieter Eykhoif en prof. Okko Bosgra bedanken voor hun bijdrage bij de totstandkoming van dit proefschrift.

Pieter Eykhoff heeft me gedurende de voorbije jaren de vrijheid gegeven om mijn eigen weg te zoeken binnen het onderzoeksgebied. Gelukkig heeft hij daarbij mijn neiging om een nog breder terrein te bestrijken ingedamd, en heeft hij me tijdig aangespoord om te beginnen met het schrijven van de dissertatie.

Okko Bosgra werd bij mijn onderzoek betrokken toen al een groot gedeelte van de resultaten in een eerste versie op papier vastlag. Zijn opbouwende kritiek in dit sta-dium en zijn nuttige suggesties voor een verdere profilering van het gepresenteerde materiaal, hebben duidelijk een positief effect gehad op het eindresultaat.

Ook de overige leden van de leescommissie, te weten prof. Michel Gevers en prof. Malo Hautus, wi! ik bedanken voor het nauwgezette lezen van het manuscript en voor hun vele suggesties voor verbetering.

Ik heb er veel waardering voor dat al deze mensen, ondanks hun overvolle agenda's, bereid zijn geweest om het proefschrift kritisch door te nemen en daar vaak zelfs veel vrije tijd aan opgeofferd hebben.

Verder bedank ik ook nag prof. Joos Vandewalle en prof. Torsten Soderstrom voor hun commentaar op gedeeltes uit de dissertatie.

Oak wil ik de vele stageaires en afstudeerders noemen die in het verleden betrokken zijn geweest bij gedeeltes van mijn onderzoek, te weten Jean Baptist Antonisse, Frank Bekkers, Emco Bos, Chen-Chao Tchang, Guus Holshuysen, Jobert Ludlage, Herman Telkamp, Wim van Beek, Ton Van den Boom, Jan Van Geloven, Wim Ver-stappen, Leon Wolters.

En uiteraard niet te vergeten Wim Beckers, systeembeheerder bij de vakgroep ER. Ontelbare keren heb ik hem lastig gevallen met vragen over het gebruik van de 11-Vax en de P.C.'s. Bijna even zovele keren wist hij mijn problemen op te lossen.

Het is ook op zijn plaats am de overige studenten en medewerk(st}ers van de vak-groep ER te bedanken, en met name Vladimir Bondarev, Ad Van den Boom, Barbara Cornelissen, Martin Driessen, Martin Klompstra, prof. Liu, Dolf van Rede, Muriel Simon, Zhu Yucai. Ik heb me altijd goed tussen hen thuis gevoeld en heb me door hun belangstelling tijdens die moeilijke laatste fase van het onderzoek gesteund geweten. I like to thank especially my room-mate Zhu Yucai for his very pleasant company. We have had many discussions from behind our Dutch-Chinese wall of books. His enthusiasm and his positive views and reactions have certainly been a stimulus in completing the thesis.

Ook denk ik nog met plezier terug aan mijn vroegere kamergenoten Andrzej Haj-dasinski en Randy Moses. Balancerend op de achterste twee paten van de stoel, ter-wijl het hoofd maar net boven de stapel boeken uitstak, hebben we vele gesprekken en discussies gevoerd.

(7)

Uiteraard brengt het vervaardigen van een omvangrijk proefschrift een immense hoe-veelheid schrijf-, type- en correctie werk met zich mee. In de eerste plaats was het nodig om me te verdiepen in de beginselen van het tekstopmaak programma Y\TEJX. Ik heb daarbij heel veel hulp van Bert den Brinker gekregen. Op hem heb ik nooit tevergeefs een beroep gedaanj hij was ten aile tijde bereid om tijd vrij te maken voor mijn vragen. Ook Paul van Loon en Piet Tutelaers zijn me van dienst geweest met informatie over Y\TEJX.

Het vele typewerk dat daarna moest gebeuren zou ik nooit op tijd hebben kunnen voltooien als ik niet de hulp van vele personen had gehad. De secretaresses mevrouw Barbara Cornelissen en mevrouw Muriel Simon hebben grote stukken "kale" tekst voor hun rekenening genomen, en hebben op velerlei manier meegewerkt bij het tot stand komen van het proefschrift. Ook Tjalling Tjalkens, toen zelf nog in het laatste stadium van zijn eigen promotie (II), bood heel genereus aan om bij te springen. Hij bewees me hiermee een ontzettend grote vriendendienst, en ik heb me verbaasd over de vingervlugheid waarmee hij vele sec ties geschreven materiaal nagenoeg foutloos heeft uitgetypt. Al deze mensen ben ik zeer dankbaar. Zonder hun had ik het nooit gered.

Ook verkeerde ik in de gunstige omstandigheid dat Leen Moelker grote hoeveelheden oud materiaal, afkomstig van de AES-tekstverwerker, kon converteren in ASCI-files op diskettes.

Aan de verdere aankleding van deze "kale" getypte tekst heb ik vele tientalle avond-uren moeten besteden. Pim Lemmens schoot mij hierbij te hulp door de PC op zijn kamer in het Hoofdgebouw s'avonds ter beschikking te stellen.

Maart was de maand waarin de correctie en de afronding van het proefschrift gro-tendeeIs moest gebeuren. Het was tevens de maand waarin ik in dienst trad bij het Centrum voor Wiskundige Methoden (C.W.M.) van het Rijks Instituut voor Volksgezondheid en Milieuhygiene (R.LV.M.) te Bilthoven. Ik ben dr. Kees van den Akker en dr. Hans Jager van het (C.W.M.) zeer dankbaar dat ze mij in die tijd carte blanche hebben gegeven om mijn werkzaamheden aan de dissertatie af te ronden, en mijn inwerktijd zo ruim een maand uit te stellen. Zonder dit genereuse gebaar zouden de spreekwoordelijke laatste loodjes niet te tillen zijn geweest.

Onder het motto "maart foert zijn staart" kwam ik in de laatste week van mll.ll.rt bij het draaien van DTEJX inderdaad nog veel onvoorziene problemen tegen die mij in een "desperate chaos" dreigden te doen verzeilen (Mr. Murphy strikes again, and again and again ... ).

Gelukkig schoot Bert den Brinker me te hulp door ruim een hele dag te assisteren bij het corrigeren en door het vakkundig tekenen van figuur 3.4.2-1. Verder was, zoaIs al vermeld, Paul Van den Hof bereid om de figuren 4.5-1 en 4.5-2 voor mij te plot-ten, en zo kon de welhaast onvermijdelijke crisis te elfder ure nog worden afgewend. "With a little help of my friends" waarbij "little" natuurlijk een understatement van de bovenste plank is ... Ik ben deze "redders in de nood" zeer erkentelijk.

(8)

iv

Tenslotte wi! ik er niet aan voorbij gaan om ook de rest van mijn vrienden, familie en bekenden te bedanken voor het feit dat ze er veel begrip voor konden opbrengen dat ik in deze drukke tijd vaak totaal onzichtbaar en onhoorbaar bleef.

Last but certainly not least, dank ik Resie. Samen met haar heb ik veel prettige momenten beleefd, en zo lukte het welhaast onmogelijke: af en toe kon ik de promotie beslommeringen vergeten, om daarna weer met frisse moed de draad op te pakken.

(9)

Instructions for the reader

Most chapters in this thesis consist of various sections. Some sections are subdivided further in subsections. The chapter number is included in the section number; the section number is included in the subsection number. Section 2.7 e.g. denotes the 7th section of Chapter 2. Subsection 2.4.5 denotes the 5th subsection of Section 2.4. The numbering of the formulas starts in each (sub)section where they occur all over again, and consists only of the formula number (between brackets). We refer to formulas in the current (sub)section by calling their formula number. Reference to formulas in other (sub)sections is made by adding the (sub)section number as a prefix.

For example, the sentence "See (7)" in Section 2.3 refers to formula (7) in this section. "See (2.2.1-5)" refers to formula (5) in Subsection 2.2.1.

The numbering of theorems, remarks, lemma's, examples etc. also starts all over again in each (sub)section. Only one counter is used for this numbering and this counter is increased with 1 each time that a new theorem, remark, lemma etc. is formulated. The (sub)section number is used as a prefix for this counter and no brackets are used (e.g. Remark 2.4.5-5, Assumption 2.6-3 etc.)

The numbering of tables and figures goes similarly. They, however, have their own counters.

Similar conventions hold for the subdivision and the numbering of the Appendices. Most currently used mathematically symbols, notations and abbreviations are listed on pages 312~317.

(13)

Summary

Mathematical models are essential tools in the study of many practical systems of various kinds (physical, chemical, biological, etc.). System identification deals with the problem of inferring these models on the basis of a priori knowledge and observed data from the system. Due to the systems complexity, as well as the incomplete avail-ability /poor quality of the observed data and the limited a priori knowledge, it is in general impossible (and even undesirable) to try to obtain an exact mathematical description of the system. This is expressed by the notion that the system does not belong to the model set. Therefore mathematical models only can describe the system approximately.

In this light it is important to specify how to approximate the system. It is obvious that in this decision the intended use of the model (e.g. for prediction, simulation, control, diagnosis) should playa major role. Other factors will also playa role, like the effort needed to obtain and use a model. Usually a trade-off will be involved between the model quality and the price/effort for obtaining a suitable model (for the intended purpose) at a low cost. This trade-off between quality and price should be reflected in the major steps which typically constitute an identification proce-dure (Le. experiment set-up, choice of the model set(s), choice of the identification method, validation of the model). All these decisions have their specific influence on the final objective of obtaining a suitable model at a low price and they should not be taken completely independently.

In this thesis we have restricted our attention mainly to the choice of the model set(s), while trying to keep track of the other items. Confining ourselves from the outset on to the study of identification by standard techniques (k-step ahead prediction error-, output error- and equation error methods), using linear, time-invariant, finite dimensional, time discrete, black-box models, the choice of the model set further re-duces to the choice of the model parametrization, Le. "How should one parametrize these model sets, and how should one choose between the various parametrized model sets (i.e. model structures), in the light of the fact that the system does not belong to the model set, and taking into account the intended use of the model".

This is the central question in the thesis, and in order to answer it we first present a mathematical framework in which we define various concepts. This framework also serves to clarify the approximate modelling aspects of the standard identification methods (Chapter 2).

Subsequently, in Chapter 3, we address the question of how to parametrize the model sets. Factors which reflect the quality and the cost of the model, such as flexibility, parsimony of parameters, computational complexity etc. are of importance. Also an important role is played by the requirement of unique parameter identifiability (U.P.1.), i.e. the parametrization should be such that the parameter estimation ren-ders a unique solution. It is studied how this requirement works out for the standard

(14)

3

identification methods mentioned, i.e. k-step ahead prediction , output error-and equation error methods.

Subsequently, confining our attention (Chapter 4) to difference equation models, we show that the common parametrizations encountered in the literature are appropri-ate indeed for output error- and k-step ahead prediction error methods, but that they are inadequate for the much used equation error methods. New parametrizations are proposed for these methods and it is studied how a priori information on causality and time delays of the system can be incorporated in these models. Furthermore we derive important asymptotical properties for these parametrized models when used in equation error identification techniques. These results provide insight in the approximation of the system by the model, and they clarify the influence of the pa-rametrization choice. Moreover they indicate that these techniques, in spite of their simplicity and computational attractiveness, should be used with care. The material on the difference equation models in Chapter 4 is completed by the presentation of new general results on the important system-theoretical notions of the McMillan degree and the dynamical indices.

Finally, in Chapter 5, we address the question how to choose between various can-didate (parametrized) model sets/structures. We argue that also here a trade-off between model quality and model price is involved. This point of view, however, has not yet fully pervaded the literature on model structure selection. In most contri-butions it is assumed that the "true" system belongs to one of the candidate model sets and, consequently, it is tried to select a "right" model structure.

In Chapter 6 we try to improve on this situation by proposing two model struc-ture selection rules which take account of the intended model use and which do not require that the system belongs to the model set. In doing so we make use of cross-validation ideas which reflect the typical fact that a. model is usually applied and assessed on data sets that differ from the one used for identification.

(15)

Chapter 1:

Introduction

1.1 Background and problem statement

Mathematical models are essential tools in the study of many practical systems/ processes. System identification deals with the problem of inferring these models on the basis of a priori knowledge of the system and observed data from the system. Due to the systems complexity as well as the incomplete availability/poor quality of these observed data and this a priori knowledge, it is in general impossible to obtain an exact mathematical description of the system (often denoted by the statement "the system does not belong to the model set"). Apart from this impossibility, it is moreover often highly undesirable to try for an exact description. For the resulting model would be very complex and costly in use, and this would prohibit further application of the model. In this light it is more natural to consider system iden-tification as approximate modelling on the basis of observed data and/or a priori knowledge.

Although the majority of the literature on system identification is still based on the (unrealistic) assumption that the system belongs to the model set, there is a growing tendency to discard this assumption and to focus on the approximate modelling aspects of identification, witness the studies on the approximation prop-erties of prediction error methods by Ljung [1976a,b], [1978), Caines [1978], Ander-son, Moore and Hawkes [1978], Kabaila and Goodwin [1980]; see also Ljung and Van Overbeek [1978], and the recent host of papers by Ljung and co-workers (cf.

Gevers and Ljung [1986], Ljung [1985a,b;1986], Ljung and Yuan [1985], Yuan and Ljung [1984,1985], Wahlberg and Ljung [19861, Wahlberg[1986, 1987]. See especially the recent book Ljung [1987]). An alternative and very attractive novel approach to approximate modelling on the basis of data, which is void of the stochastical as-sumptions which are commonly used in the more convential approaches mentioned before, was presented recently by Willems [1987].

In this thesis we will adhere to the viewpoint that the system typically does not be-long to the model set, and that therefore system identification should be considered as approximate modelling on the basis of the observed data and a priori knowledge. Now it is obvious that the way in which this approximation will take place, and its final outcome, is determined by the four basic steps which typically constitute the system identification procedure:

1. Choice of the experimental set up

Choice of the signals to be measured, sampling rate, input (test) signals, feed-back configuration, data-acquisition and -processing etc.

2. Choice of the model set(s)

(16)

1.1 Background and problem statement 5

approximation of the system?

3. Choice of the identification method/criterion

Which method should be used for the selection of a model in these model sets, and how should this selection be performed numerically (which algorithms)? 4. Validation of the model

Is the obtained model good enough (for its intended use) 1 H not reconsider the procedure.

The respective choices are illustrated in Figure 1.1-1 (d. figure 1.11 in Ljung [19871).

:---Experimental

I---

Data

I---Design _C 0 m p Model set u Choice

r----

t

f----

Validation a t i 0 Choice of n Identification

I---Method r -A priori knowledge; Objectives

Figure 1.1-1: The various steps of the identification pro-cedure

O.K.

No t K. O.

By a judicious choice of these items we can influence the feasibility to obtain a suit-able (approximate) model at a low price. The decisions in (1)-(4) cannot and should not 'be'seen completely independent from each other. They should, instead, be bal-anced against each other, taking the a priori knowledge and the intended model use (e.g. for prediction, simulation, control, diagnosis. monitoring etc.) as our main guiding principles. Notice that usually a trade-off between model "quality" and model "price" will be involved in these decisions, in order to obtain a suitable (ap-proximate) model (at a low price) (d. Ljung [1987]).

(17)

we confine our attention mainly to the important problem (2) of choosing appro-priate model sets, while keeping track of the possible connections with the other items. Besides we will restrict ourselves mainly to standard time-domain identifica-tion methods, like output error-, k-step ahead predicidentifica-tion error- and equaidentifica-tion error methods, Le. O.E.M., k-step ahead P.E.M. and E.E.M. (see item (3)).

Considering item (2) in more detail we can typically divide it into three subproblems: (1) Choice of a model type

There exists a wide variety of model sets which can be characterized by many adjectives such as static-dynamic; linear-nonlinear; deterministic-stochastic; lumped parameters - distributed parameters; discrete time - continuous time; time invariant - time variant; causal - non causal. Besides one has to decide whether one can infer a suitable model on the basis of a priori knowledge and engineering insight, leading e.g. to a model containing some unknown physical parameters (Le. grey-box modelling). An alternative is to use e.g. standard linear models whose parameters need not have physical meaning but merely serve as means to obtain a good fit to the data (Le. black-box modelling). In this thesis we will restrict ourselves mainly to the important class of linear, time-invariant, time-discrete, finite-dimensional, black-box models, which are appropriate tools for approximating many practical processes.

(ii) Choice of a model representation

The above mentioned model class can be represented in various ways, e.g. in state space forms; transfer function forms; impulse response forms; difference equation forms.

(iii) Choice of the model parametrization/model structure

The model class thus represented is still rather large (e.g. all finite dimensional linear state space models), and should therefore be subdivided in subsets of restricted size (e.g. state space models of order n for n 0,1,2,··' etc.) In order to enable a choice of a suitable model from any of these subsets by means of a specific identification method sub (3), one introduces numerical parameters for specifying models from these sets. Usually, especially for MIMO-systems (i.e. Multi Input Multi Output systems), one needs several parametrizations (each characterized by specific structure indices) to adequately describe any of these model subsets. Consequently, at this stage one is confronted with the following questions:

-a- Which model sets should be considered? (choice of the model size) -b- How should one parametrize these sets?

-c- How should one choose between these candidate parametrized model sets/model structures? (model structure choice)

(18)

1.2 Overview of the thesis and main contributions 7 Having sketched in the foregoing the background we finally come up with the problem that will be central in this thesis:

How should one parametrize model sets for multivariable system identifi-cation, and how should one choose between parametrized model sets/model structure8, taking into account the intended use of the model and the fact that the system does not belong to the model set.

In the next section we will indicate how the study of this problem is reflected in the thesis.

1.2 Overview of the thesis and main contributions

In the thesis we will address various sub-problems of the above given problem state-ment:

(a) Definition of fundamental concepts

What is a model, model set/model structure? What is a parametrization? These items are treated in Chapter 2 and in Section 3.3.

(b) How to parametrize models?

Which requirements should be fulfilled (Chapter 3)1 Which possibilities are available for parametrizing models and what are their properties? Attention will be focussed on the important class of difference equation models (Chapter

4).

(c) How to decide between various (parametrized) model sets

I

model structures

First some general considerations on this topic are given and a review of the existing literature is presented (Chapter 5). Subsequently, on the basis of cross-validation ideas, we propose two new methods for making a selection between various model sets, taking the intended use of the model into account (Chapter 6).

This enumeration shows that the thesis consists of three parts.

In the first part the concepts which playa role in identification are defined. The material is presented in Chapter 2 and encompasses most standard time domain identification methods. In proposing our conceptual framework we aim at a more explicit treatment of the approximate modelling aspects of these identification meth-ods in a rigorous mathematical style. Central is the (pragmatic) notion of Residual

Generating Model (R.G.M.) which reflects the central role of the modelling error (model residual) in these methods and which serves to express their approximate modelling aspects explicitly (Section 2.2). This notion is strongly related to Ljung's "predictor model" (cf. Ljung [1987]). Our treatment is however more general and is presented in a mathematical somewhat more stricter style.

(19)

While the notion of R.G.M is a natural one in the context of approximate mod-elling on basis of measurement data (Le. identification; "from data to models"), we next take a somewhat opposite point of view, and look for descriptions which tell us how the data are generated, Le. which exact dynamical relationships will hold between the data ("from models to data"). [This will be described by the concept of Data Generating Model (D.G.M.j, which is inspired on Gevers and Wertz [1987a,bj who use a less general concept, and is related to Willems' definition of model (cf.

Willems [1986b]).] Since we believe that the observed phenomena (data) are far too complex to be modelled in useful exact mathematical terms, this viewpoint has to be considered as a thought-experiment rather than as a realistic premise. The major motivation for introducing this notion is its use in theoretical analysis (where we assume that some underlying D.G.M. generates the data) and the fact that most R.G.M.'s are "deduced" from D.G.M.'s.

As an illustration of the abstract definitions in Section 2.2 and Section 2.3 we next present in a unified way (see Section 2.4 and Section 2.5) various examples of linear time-invariant D.G.M.'s (e.g. state-space models, difference equation models etc.) and R.G.M.'s (e.g. k-step ahead prediction error models, output error models, equa-tion error models). This material is rather standard and contains many well-known properties which are listed for reasons of completeness and for further reference. In the subsequent Section 2.6 we propose a general asymptotic and (pseudo) stochas-tic framework in order to study the approximate modelling aspects of standard time domain identification methods which use linear, finite-dimensional models. Expres-sions are derived for the quality of the identified models which can be used for studying how the quality can be affected by choosing the various design variables. These results are derived for MIMO systems and are generalizations of the recent·

8I80 results in Gevers and Ljung [1986], Wahlberg and Ljung [1986] and Ljung [1987].

Finally, In Section 2.7, we critically review the presented material and briefly discuss various alternative approaches to identification.

The second part of the thesis is devoted to the question of "how to parametrize mod-els for identification purpasesf". General considerations are presented in Section 3.2, where various requirements are formulated which should be fulfilled in order to obtain a suitable approximate model at reasonable costs. Special attention is given to the requirement of unique parameter identifiability (U.P.I.), ensuring that the identification method gives a unique (optimal) solution. Although this concept of U.P.I. has in fact been defined and used for a long time (cf. Bellman and Astrom [1970]), its utmost consequences have not been fully pursued in literature. After presenting various abstract definitions on parametrizations in Section 3.3, we there-fore study in Section 3.4 more deeply how unique optimal models (U.P.!.) can be achieved for O.E.M., k-step ahead P.E.M. and E.E.M. This leads to new insights and results, indicating that the traditional treatment of identifiability for equation error identification methods is inadequate. Our observations are in agreement with

(20)

1.2 Overview of the thesis and main contributions 9 the recent results of Van den Hof [1987,1988a,b,cj on this item.

Subsequently we confine our attention to the important class of difference equation models, omitting treatment of other relevant model classes such as state space mod-els (cf. e.g. Gevers and Wertz \1987a,b] for more information on parametrizations of state space models). Using a general framework, a neat overview is presented of various commonly used parametrizations for difference equation models (Subsec-tion 4.3.1 and 4.3.2). These parametriza(Subsec-tions are suitable for O.E.M. and k-step ahead P.E.M. Although these common parametrizations also render unique models for E.E.M., we will show that their use is, however, not appropriate in this context. Using the results and proposals from Section 3.4 two new U.P.I. parametrizations for E.E.M. will be proposed (cf. Subsection 4.3.3) which are more suitable in this context.

When using these difference equation models for modelling the I/0 relationship, we are confronted with the important question of how to incorporate (a priori) information on the causality and the time-delays of this 1/

°

relationship in the (parametrized) models. This question is addressed in Section 4.4.1

In the subsequent Section 4.5 the use of difference equation models in equation error methods is studied in more detail. 2 _{Some new important asymptotic properties of}

these models are derived for the typical situation where the system does not belong to the model set. These results give further insight in the approximate modelling aspects of these techniques, indicating their possible weaknesses, and highlight the influence of the parametrization choice.

The material on the difference equation models is further completed by the pre-sentation of general new results on the important system-theoretic notions of the McMillan degree and the dynamical indices (cf. Section 4.2).

Concerning the material in the second part of the thesis, as outlined above, we has-ten to say that we do not give absolute and definite considerations (if this would be possible at all) on which kind of parametrization to choose for MIMO identification. We confine ourselves mainly to listing various possibilities for parametrizing models and we present some general guidelines which should play a role in choosing the parametrization (see e.g. Section 3.2).

After one has chosen the kind of parametrization to be used, one is confronted with the problem of how to decide between the various parametrized model sets/model structures. This problem is addressed in the third, and final, part of the thesis. A general consideration on this problem, and an overview of the pertinent literature is given in Chapter 5. It appears as a rather striking fact that most techniques for making this choice are based on the unrealistic assumption that the system belongs to one of the considered candidate model sets, and do not take the intended use of the model into consideration. We will take a more realistic viewpoint and

as-IThe material of Section 4.4 is a slight extension of Janssen [19871.

:lThe material of Section 4.5 is a slight extension of the joint work with ir. Paul Van den Hof; cf. Van den Hof and Janssen [1985,1986,1987].

(21)

sume that the system can only be modelled approximately by the candidate model sets/model structures. Against this background it is more useful to consider the model structure selection as choosing a model structure which will enable an accept-able approximation of the features of the system which are important w.r.t. the intended application. Taking moreover into consideration that usually models are applied to data different from the ones used for identification, we use cross-validation ideas to propose two new general model structure selection methods which take the intended use of the model into account (Chapter 6).3 Moreover we establish, under somewhat restrictive conditions, asymptotic equivalences between our methods and some well-known model structure selection rules.

Finally in Chapter 7 we conclude our presentation with a review of the obtained re-sults and indicate directions for further research. The thesis is further supplemented with various appendices containing definitions, results and proofs.

SThe material of Chapter 6 is a slight revision of the joint work with dr. Petre Stoic a, prof. Torsten Soderstrom and prof. Pieter Eykholf. See Janssen et al. [1987J.

(22)

2.1 Introduction 11

Chapter 2:

Models, criteria and approximate modelling aspects

of identification

2.1 Introduction

In Chapter 1 we argued that in general it is impossible (and often even undesirable) to obtain an exact mathematical description of the properties/behaviour of a real system. Based on this observation we consider identification as approximate mod-elling on the basis of experimental data. It will be of crucial importance to specify which aspects of the system will be approximated, how this will be done, and what we mean by an (approximating) model. These fundamental questions have been considered in literature at various levels of depth and rigour.

Confining ourselves to identification of discrete time systems by parametric time-domain methods, a standard way to obtain approximate models can roughly be stated as follows: Specify a modelling error/model residual, obtained by performing specific operations on the experimental data, and try to make this residual "small" in some sense, e.g. by minimizing a certain scalar measure of the residual. The choice of this model residual and this measure should reflect the important aspects of the system which we want to approximate and the intended goal(s) of our modelling efforts. This approach is advocated and formalized amongst others by professor L. Ljung (see e.g. Ljung [1978], Ljung and Soderstrom [1983], Ljung [1987]). He defines a model as "a rulE to make some sort of inference about future outputs of the system on the basis of observations of previous data" (cf. e.g. Ljung and Soderstrom [1983], page 71; in Ljung [1987] this concept is formalized in mathematical terms) and he uses the term prediction error identification methods to denote a wide class of identification methods encompassing output error methods, equation error methods, maximum likelihood methods etc.

Recently an important alterna.tive contribution to put a solid foundation under the problem of identification/approximate modelling has been given by professor J.C. Willems who addressed various fundamental questions in an elegant and rig-orous style. He presented a neat set-theoretical for discussing dynamical systems (Willems [1986al) and developed concepts and terminology for exact modelling on the basis of experimental data (Willems [1986b]). Starting from exact modelling he proposed a methodology for approximate modelling based on the concepts of misfit and complexity. Using a misfit function based on equation errors he finally presents a new approximate modelling procedure, serving mainly the purpose of description (Willems [1987]). Heij [1987J used Willems' ideas to develop an approximate mod-elling procedure for prediction purposes.

(23)

on Gevers and Wertz [1987a,b], Hanzon [1986]), we present in this chapter a concep-tual framework for time-domain identification of discrete time systems. The major motivation behind our approach is threefold:

1. First we like to formalize and make explicit the approximate modelling aspects of identification.

2. Secondly, we like to have a general framework in which the approximate mod-elling features of various identification methods can be easily discussed and studied.

3. In the third place, we want to pave the way for some critical reconsiderations of the items of parametrizations and identifiability (cr. Chapter 3).

Needless to say, since these items are our main objectives, our contribution must necessarily be considered as a somewhat incomplete attempt to put some mathe-matical basis underneath the field of identification.

In order to reach the aforementioned objectives we use the concept of modelling error (model residual) as a central tool. This concept in fact serves to explicitly express how the system is approximated by the model on the basis of the data, and forms the basis for the pragmatic definition of a residual generating model (R.G.M.) (see Section 2.2). The definition of a R.G.M. is strongly related to Ljung's definition of predictor model (cr. Ljung [1987]), but our approach is more general and is stated mathematically somewhat more rigorously. [Remark: We like to point out that Willems' definition of model (cf. Willems [1986bJ) is not completely appropriate for our purposes, since the approximation aspects do not show up in his definition. They come in indirectly by use of the concept of misfit (cf. Willems [1987]), which in fact uses a definition of a particular modelling error, namely the equation error]. Subsequently in Section 2.3 we introduce the notion of data generating model (D. G.M.) (see Gevers and Wertz [1987a,bJ), which is more in accordance with Willems' defi-nition of a model. This concept in fact describes how the data (e.g. output data) are generated as a function of source signals (e.g. input data and (possibly) some auxiliary variables), and is more of theoretical value than of practical value. In fact, we believe that practical systems are usually far too complex to allow for a useful exact mathematical description. However, for the sake of theoretical analysis, we often assume that the behaviour of the system is generated by an underlying data generating model. Although this assumption is somewhat aside reality, the essential features of the system can often be suitably approximated by a D.G.M., and the results derived from a theoretical analysis based on this assumption will then give good insights in the real situation. Another rationale for introducing the concept of D.G.M. is given by the fact that R.G.M's are often derived/proposed (initially) on the basis of D.G.M.'s

Various representations of linear D.G.M.'s and their interrelationships are subse-quently presented in Section 2.4. This material is rather standard; various well-known properties are stated for reasons of completeness and for ease of reference.

(24)

2.2.1 Definitions 13

Next, using the material of Sections 2.2, 2.3 and 2.4, we give in Section 2.5 vari-ous examples of R.G.M's and present varivari-ous well-known time-domain identification procedures in a unified way.

Using the general framework of Section 2.5, we discuss in Section 2.6 the approxi-mate modelling aspects of the presented identification methods in more detail, using asymptotical (pseudo) probabilistic assumptions. This leads to some useful general expressions for the quality of the identified model which clarify the influence of the various design variables. The material in Sections 2.5 and 2.6 is an extension of the

SISO results in Ljung [1987], Gevers and Ljung [1986], Wahlberg and Ljung [1986]

to the multivariable situation.

Finally in Section 2.7 we conclude with a critical review of the presented material and briefly discuss some alternative approaches to identification/approximate modelling.

2.2 Residual generating models (R.G.M.)4

A standard approach to approximate modelling on the basis of experimental data is to specify a model residual (modelling error, e.g. output error), consisting of certain operations performed on the experimental data, and trying to make this residual "small" (in some sense).

In this section we will mathematically formalize the first part of this procedure (con-sisting of the specification of a modelling error) by introducing the new concept of residual generating models (R.G.M.). This notion clearly expresses the approximate modelling features of identification, since it in fact defines the model residual (mod-elling error), i.e. a quantity expressing which part of the data is left unexplained by the model. Various useful properties (like linearity, time invariance) for R.G.M.'s will be defined (Subsection 2.2.1). Subsection 2.2.2 contains a discussion on the stochastic situation, where the measurement data are assumed to be realizations of underlying stochastic processes. In Subsection 2.2.3 finally, we define some addi-tional concepts which are related to exact modelling and to the somewhat unrealistic property that the "system is representable within the model set".

2.2.1 Definitions

We first present some simple examples, serving to motivate and illustrate the forth-coming definitions:

Example 2.2.1-1 Suppose that we have measurements available of the input signals

u(t) E IR" , and the output signals y(t) E IRq, (p, q E IN) , for t :::: 0,1,2, ... , N (N E IN). Let now A E IRnxn , B E nnxp, C E nqxn, D E IRqxp, Xo E IRn for certain n E IN, and generate

the signals x(t) E IR'" y(t) E IRq as follows:

the initial stage of defining this concept I benefitted from discussions with ir. Paul H.M. Van den Hof.

(25)

1

x(t

+

1) Ax(t)

+

Bu(t) x(O) Xo

yet)

ex(t)

+

Du(t)

(t 0, 1,Z, ... , N) (1)

Define

~(t) := yet)

yet)

(t

=

0, 1, Z, ... , N) (Z)

The quantity ~(t) is called an output error, and indicates the difference between the measured output yet) and the simulated output

yet)

generated by the expressions (1). It is often used as a basis for identification methods (output error methods, cf. Ljung and Soderstrom [1983J,

Ljung [1987], Soderetrom and Stoica [1987J; see also Section 2.5).

•

Example 2.2.1-2 Suppose that we have available the input and the output signals u(t) E

IR" and yet) E IRq, (p, q E N) respectively, for t

=

0, 1, 2, ... , N (N EN). Define the quantity ~(t) E IRq by

~(t) P(Z) yet) - Q(Z) u(t) (3)

where P(z) E IRqXq[zj is a nonsinguJar polynomial matrix and Q(z) E IRqxP[zj is a polynomial matrix given by (n, mEN)

P(z) Po zO

+

P1 zl

+ ... +

P_nzn

Q(z)

=

QOZo+Q1z1+'''+Qmzm

(4) (5) The symbol Z denotes the forward time-shift operator given by Zy{k):= y(k+ 1). [Remark: In order to avoid confusion we shall denote the forward shift operator by a capital letter Z. The indeterminate variable appearing in the polynomial matrices will be denoted by a lower-case letter z.]

Therefore (3) compactly denotes that ~(t) is determined by a set of forward difference equa.-tions as function of the inputs and outputs. ~(t) is called the (forward) equation error, and is used as a basis for equation error identification methods. (See Section 2.5, cf. Ljung and Soderstrom [1983], Ljung [1987], Soderstrom and Stoica [1987]). Notice that we can only compute the signal ~(t) given by (3), on the interval [0, N -

mol

where mo is the maximal degree of all the entries of P(z) and Q(z). In order to extend the definition of ~(t) on the interval [0,

NJ,

we have to specify various values for u(t) and yet) for t > N.

•

Example 2.2.1-3 We have available the input and output signals u(t) E IRP, yet) E

IRq, (p, q E N) for t 0,1,2, ... , N (N EN). Now define the quantity e(t) E IRq by

e(t) := P(D) yet) Q(D) u(t) (6)

where P(d) E IRqxq[d] is a nonsingular polynomial matrix and Q(d) E IRqx"[dj is a polynomial matrix given by ( ii, mEN)

P(d) Po <f!

+

PI d1

+ ... +

Pn:dn: Q(d) = Qo<f!+Q1d1+"'+Qm:dm:

(7) (8)

(26)

The symbol D denotes the backward time-shift operator, given by Dy(lc) ;= y(k-l).[Remark: In order to avoid confusion we shall denote the backward shift operator by a capital letter D. The indeterminate variable appearing in the polynomial matrices will be denoted by a lower-case letter d.]

Therefore (6) compactly denotes that e(t) is determined by a set of backward difference equations as function of the inputs and outputs. e(t) is called the (backward) equation error and is used as a basis for equation error identification methods (see Section 2.5, cf. Ljung and Soderstrom [1983], Ljung [1987], Soderstrom and Stoiea [1987]). Notice that the signal e(t) can only be computed on the interval [ml' N] where ml is the maximal degree of all the entries of P(d) and Q(d). In order to extend the definition of E(t) on the interval [0, Nj, we have to specify various "initial" values for u(t) and yet) for t < 0 .

•

The previous examples illustrate several ways of associating a sequence of error (residual) signals to a sequence of input-output data signals. Motivated by these ex-amples we formalize this in the following definitions, restricting ourselves to discrete-time models (Remark: For arbitrary sets A and B the symbol AB denotes the set of all mappings {! : B ... A}; see the list of symbols):

Definition 2.2.1-4 [Data sequence space]

Let TD C 7]" dE IN and [) C (JRd)TD. [) is called the data sequence space.

•

Definition 2.2.1-5 [Residual sequence space]

Let TR C 7]" rEIN and

R

C (JRrfR .

R

is called the residual sequence space.

•

TD denotes the time-axis on which the (measured) data signals are defined. TR denotes the time-axis on which the model residual signals (modelling errors) are defined. Now we define an R.G.M. as a "rule" prescribing how the model residuals are generated from the data signals:

Definition 2.2.1-6 [Residual generating model (R.G.M.)l Let [) and

R

be as given in Definition 2.2.1-4 and Definition 2.2.1-5. A residual generating model (R.G.M.) is a mapping 'P : [) ...

R.

•

Remark 2.2.1-1 -a- With some abuse of terminology we will call an element DE D (re-spectively R E R) a data 8equence (respectively a. residual sequence), and will denote it as

D(.) , D : TD -+ ltd or (D(t))tETD (resp. R(.), R: TR -+ IS: or (R(t))tETn) .

-b- Reconsidering the Examples 2.2.1-1, 2.2.1-2 and 2.2.1-3 in the light of these definitions, shows that the data sequence space f) C (lRd)TD often consists of " sequences" D ; TD ... IRd of measured input-output signals, e.g. D(t)

=

Vee (u(t), y(t)),d

=

p+q and TD {O, 1, ... , N}.

(27)

(For a definition of the Vee-operation see Appendix E). The output errors, respectively forward or backward equation errors in these examples constitute the residual sequence

R : TR ...

IR:

(so r"" q and TR C TD)'

-c- Notice that Definition 2.2.1-6 of an RG.M. is very general: an R.G.M. 'f/ associates to any data sequence DE [) a residual sequence R:= 'f/(D) E R.. No specific requirements are imposed on the way in which this is done (in identification practice the residual sequence R is usually constructed by performing (in a recursive way) some algebraic operations on the data sequence D).

Ljung's definition of (predictor) model is more restrictive in this respect (cf. page 71 in Ljung and Soderstrom [1983]; page 134 in Ljung [1987]), but it can easily be linked to our defini-tion. Ljung defines a (predictor) model as a rule to make inference, based on observations of previous data, about future outputs of the system (see Ljung [1987], page 134, for a more formal description). Denoting such an inference, or "prediction" by

yet),

the "prediction error" yet)

yet)

can be formed. As such, Ljung's definition can be regarded as a special case of our more general notion of RG.M.

-d- Important questions related to above definitions are: How should we define D7 (and of lesser importance: how should we define R.7). Which data sequences DE (JRd)TV should be taken into consideration (certainly only those for which 'f/(D) exists) and what properties will the range 'f/(D) consequently have? A further item of interest is : How to deal with stochasticity? In case the measured data sequences are realizations (sample pa.ths) of un-derlying stochastic processes, the RG.M. will associate to each sample path (D(t»)tETv in [) a residual sequence (R(t)}tETn' An important question which will be discussed in Subsec-tion 2.2.2, is : Can these residual sequences be considered as sample paths of an underlying stochastic process?

•

In Definitions 2.2.1-4/2.2.1-6, no further specification wa.s given on the choice of the data-sequence space and the residual sequence space. In the next examples we will present various spaces in which these signals are assumed to lie.

Example 2.2.1-8 If we consider s-dimensional signals defined on a finite time-interval T"" {D,I,2,···,N I}, a natural choice for the signal space S is S = (JRdV. Bearing in mind that we often use weighted sum-or-squares criteria for identification, we define the norm:

{

1 N-l }1/2

1\ s IIn:= N

t;

sT (t) fl set) (9)

where fl > 0 E IRdxd is a positive definite matrix. S is easily seen to be a real Banach space with norm

II·

lin.

Example 2.2.1-9 If T Zl+, things are less ideal. The set

1 N-l

W(Zl+,JRdjfl):= {s E (JRd}7.t+

I

lim -

L

sTet) fl set) < oo} N-+oo N 1=0

•

(28)

2.2.1 Definitions

17

(where 0 >

°

E RdXd₎_{is unfortunately not a}_linear_{subspace of}_(Rd

)7h+ and has also other unfavourable properties (see Appendix B, Remark B-1). Therefore we consider the set

(11) with

{

I N-l }1/2

IIsl/n:=

lim sup N

L

sTet) 0 set)

N .... oo t=o

(12)

Note that fl plays no role in the definition of B(:lZ+,Rd_)._{See Remark B-1(a) in Appendix}

B for an explanation.

B(:lZ+,Rd₎_{is a Banach space with (semi)-norm}

_{II .}

₁₁₀

_(Sl_and_S2_E_B(:lZ+,Rd₎_{for which}

II

S1 - S2

Iln=

°

are considered as identical; see Theorem B-2).

Central in our study will however be the subset:

(Ift+r <O,then s(t+r) 0)

The signals s E Q{2Z+ , Rd) are called quasi-stationary signals and we define:

1 N-l

R.(r):= lim -

L

s(t+ r) sTet) N->oo N 1=0

(r E :lZ) (14)

Notice that for s E Q(:lZ+) Rd)

1 N-l

lim N

L

sT (t) 0 set)

=

trace (R.(O) 0) = trace (0 R.(O» N ... oo 1=0

(15) exists. Considering moreover the subset (the subscript" abs" denotes" absolutely summable" ;

II . II

denotes an arbitrary matrix norm)

+00 Qabs(:lZ+,Rd):= {s E Q(:lZ+,Rd)

I

L

II

R.(r)

11<

Do} (16) r=-r;x.> +00 <l>.(w):=

L

R.(k)exp(-jwk) (17) k=-oo

<l>.(w) is called the spectrum of s. Extension of this concept to general quasi-stationary signals is possible, cr. Remark B-6. The spectrum fulfils:

• <l>.(w) =: <l>.(-w)

=

<l>;(w) (18)

where

A

denotes complex conjugation of the matrix A. So <l>.(w) is a Hermitian matrix (a matrix A E([)nxn is called Hermitian if A* :=

AT

is equal to A).

(29)

•

<I>.(w) :?: 0 (19) Le. a* <I>,(w) a:?: 0 for all a E(Jjd.

1

1'-Ra(r) = -2 exp(jwr) <I>.(w)dw 1r _".

•

(20)

Moreover we will call two signals {sl(t)he7Z+ and {s2(t)he7Z+ jointly quasi-stationary if the

composed signal [

:~m

]

is quasi-stationary.

Similar definitions and properties hold for signals defined on T

=

'll..

•

The (asymptotic) framework which we obtain by considering the above given quasi-stationary signals will be called pseudo-stochastic/pseudo-probabilistic framework. Moreover we will consider signals which are embedded in a probabilistic framework.

Example 2.2.1-10 Considera d-dimensional stochastic process {set) he7Z for which IE set) and IEs(t) sT(r) (t,r E 'll.) exist, and suppose that

1 N-l

R.(r):= lim

NL

IEs(r+t)sT(t)

N .... oo t=o

(21) exists for any r E 'll. (IE denotes the expectation operator). Such a process is called a

gen-eralized weakly stationary stochastic process. Moreover we will call two stochastic processes {sl(t)he7Z and {s2(t)he7Z jointly generalized weakly stationary if the composed process

{ (

:~

m ]}

tE7Z is generalized weakly stationary.

These concepts extend the notion of (jointly) weakly stationary stochastic processes (Le. pro-cesses for which IE set) is constant and IE s(t + l)sT (r + I) IE s(t)sT(r) for alII E 'll.), and allow for the presence of nonstationary

I

deterministic components in stochastic signals (the expectation is taken with respect to the stochastic components of set)}.

In case that +00

L

II

R.(r)

11<

00 (22) r-=-oo we define +00 4>.(w)=

L

R.(k)exp(-jwk) (23) k;-oo

<I>.(w) is called the spectrum of s(.). Extension of this concept to the situation where (22) does not hold is possible, cf. Remark A-5. The spectrum fulfils also the properties mentioned in (18)-(20).

•

Before proceeding with specifying various properties for R.G.M.'s we like to comment on the (asymptotic) frameworks presented in Examples 2.2.1-8/2.2.1-10:

(30)

Comment 2.2.1.11

In our theoretical analysM of identification methods we will confine ourselves mainly to asymptotic situa.tions. Reason for this is that consideration of the finite sample case usually gives rise to extremely complicated expressions. However, in the limit

(N -+ 00) these become more tractable.

In these asymptotic situa.tions usually average mean square expressions will be in-volved, and therefore consideration of quasi-stationary signals or generalized weakly stationary stochastic processes is most adequate. Since the second-order moment properties of these signals are similar, the results for the deterministic case (quasi-stationary signals) and the stochastic case (generalized weakly (quasi-stationary stochastic processes) will be similar.

Finally we like to point out some further consequences of working within the above mentioned (pseudo)-probabilistic framework:

(1) We only consider the (asymptotic) second-order moment properties of the sig-nals. The influence of transient components in the signals will be no part of the analysis.

(2) By considering quasi-stationary signals we confine ourselves to signals having finite power and infinite energy. Signals with finite energy do not enter the picture.

(3) Second-order moments of signals only give partial information on their char-acteristics. Signals which have the same second-order moment properties can show a completely different behaviour (compare figure 2.5 and figure 2.6 in Ljung [1987]).

•

After this lengthy discussion on signal spaces we proceed with classifying the R.G.M.'s further by introducing the straightforward notions of linearity and time invariance. (Various examples of R.G.M.'s will he presented in a unified way in Section 2.5).

Definition 2.2.1-12 [Linearity for an R.G.M.]

Let f) C

(lRdfD

and R

C (lR'fn

he vector spaces over IR. Let I{! : f) - t R be an R.G.M. as defined in Definition 2.2.1-6. I{! is called a linear R.G.M. if I{! is a linear mapping.

Definition 2.2.1-13 [Time invariance for an R.G.M.]

Let I{! : f) -+ R be an R.G.M. Define:

8'1> := {(D, I{!(D)) IDE f)} C f) X R

•

(31)

Let Tv C 7], be an additive semigroup (i.e. [tbt2 E Tv] =?- ttl

+

t2 E TvD, and let TR E 7], be Tv-additive. (a subset T C 7l is called Tv-additive if ttl E T,t2 E Tv]

=?-[h

+t2

E TJ).

The R.G.M. 'P : [) -+ R is called a time-invariant R.G.M.if for all

t

E Tv

(25)

where ul _{denotes the t-shift: i.e. let}_T_C

7], be Tv-additive, and

f :

T -+ F, then

ut

_{f :}

_T

-+ F is defined for t E Tv as (ut f)( r) := f( r

+

t).

[Remark: In expression (25) ul

[8'PJ

_denotes_{(ut _D,ul _'P(D))_{IDE [)}.]}

•

Remark 2.2.1-14 -a- The conditions that Tv is an additive semigroup and that Tn is Tv-additive are imposed to ensure the existence of the shifted signals qt D and qtcp(D). -b- The above definition is rather restrictive, since we require that Tv is an additive semi group ( and Tn is Tv-additive). Practical situations where Tv is a finite time set are therefore excluded. This seems to be a severe restriction, but by adaptation of the above definition (compare the shifted sequences with the truncated finit,~ sequence5) these situations can also be treated. We will not elaborate on this (cf. Heij [1986]). .

•

The R.G.M.'s presented in Example 2.2.1-2 and Example 2.2.1-3 are linear and time invariant (if e.g. Tv = 7],+). However the R.G.M. in Example 2.2.1-1 will not have these properties if x.

i

O. Therefore we now extend our definitions such that R.G.M.'s with initial conditions, like in Example 2.2.1-1, can be considered as linear and time-invariant.

Inspired by Hanzon [1986] we consider the R.G.M.'s

'PiJ : [) -+ R ({1 E I)

where I is an index set, called the set of initial conditions, and define: Definition 2.2.1-15 [R.G.M. with s.i.e.]

Let 'PiJ : [) -+ R be an R.G.M. with initial conditions (1 E I.

The mapping TP : I x [) -+ R defined by

TP({1, D) := 'Pp(D)

/3

E I, DE [)

is called an R.G.M. with set of initial conditions (s.Le.).

(26)

(27)

•

Notice that we only consider R.G.M. 's 'PiJ in (26) and (27) having a domain [), which is independent of {1 E I. I is usually chosen as a subset of some Euclidean space

On model parametrization and model structure selection for identification of MIMO-systems

On model parametrization and model structure selection for

identification of MIMO-systems

On Model Parametrization and

Model Structure Selection

for Identification of

MIMO-Systems

Proefschrift

Peter Hubertus Maria Janssen

door de prornotoren

prof. dr. ir.

P.

Eykhoff

en

prof. ir. O. H; Bosgra

Coprornotor dr. ir. A. A. H. Darnen

Janssen, Peter Hubertus Maria

On model parametrization and model structure selection for

identification of MIMe-systems / Peter Hubertus Maria Janssen.

[S.l. : s.n.]. Fig., tab.

Proefschrift Eindhoven. - Met lit. opg., reg.

Contents

Instructions for the reader

Summary

Chapter 1:

Introduction

1.1 Background and problem statement

I---

r----

f----

1.2 Overview of the thesis and main contributions

I

°

Chapter 2:

Models, criteria and approximate modelling aspects

of identification

2.1 Introduction

2.2 Residual generating models (R.G.M.)4

1

+

+

yet)

+

yet)

=

yet)

•

=

+

+ ... +

=

mol

NJ,

•

+

+ ... +

•

•

R

R

•

R

R.

•

=

=

IR:

yet),

yet)

•

t;

II·

lin.

I

L

•

17

°

IIsl/n:=

L

_{II .}

₁₁₀

_{f :}