This manuscript has been reproduced from the microfilm master. UMI films the
text directly from the original or copy submitted.
Thus, som e thesis and
dissertation copies are in typewriter face, while others may be from any type of
computer printer.
The quality o f this reproduction is dependent upon the quality o f the copy
subm itted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleedthrough, substandard margins, and improper alignment
can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete manuscript and
there are missing pages, these will be noted. Also, if unauthorized copyright
material had to be removed, a note will indicate the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning
the original, beginning at the upper left-hand com er and continuing from left to
right in equal sections with small overlaps. Each original is also photographed in
one exposure and is included in reduced form at the back of the book.
Photographs included in the original manuscript have been reproduced
xerographically in this copy. Higher quality 6” x 9” black and white photographic
prints are available for any photographs or illustrations appearing in this copy for
an additional charge. Contact UMI directly to order.
UMI
Bell & Howell Information and Learning
300 North Zeeb Road, Ann Arbor, Ml 48106-1346 USA 800-521-0000
by
H arold Todd W areliam
B.Sc.h.. Memorial University o f Newfoundland, 1985 B.A., Memorial University o f Newfoundland, 1986 M.Sc., Memorial U niversity o f Newfoundland. 1993
A D issertation Subm itted in P a rtia l Fulfillment of the Requirem ents for th e Degree of
D O C T O R O F PH IL O SO PH Y in the D epartm ent of C om puter Science
We accept this d issertatio n as conforming to the required stan d a rd
Dr. M ichael R. Fellows, Supervisor (D epartm ent of C om puter Science)
Dr. Valerie King, D epartm ental M ember (D epartm ent of C om puter Science)
_______________________________________
Dr. Frank Ruskey, D epartm ental M ember (D epartm ent of C om puter Science)
Dr>~JEwa C ^ y k q W^Ica^iggins. O utside M em ber (D epartm ent of Linguistics)
Dr. Bruce W atson, E xternal Exam iner (R ibbit Software, Kelowna, BC, Canada)
© Harold T odd W areham , 1999
U niversity o f V ictoria
A ll rights reserved. This dissertation may n o t b e reproduced in whole or in p a rt, by photocopy or other means, w ith o u t perm ission of the author.
Supervisor: Dr. Michael R. Fellows
ABSTRACT
Many com putational problem s are iV P-hard and hence probably do not have fast. i.e.. poly nomial time, algorithms. Such problems may yet have non-polynom ial time a l g o r i t h m s and the non-polynomial tim e complexities of these algorithm s will be functions of particular as pects of th a t problem, i.e.. the algorithm 's running time is u p p er bounded by f{ k )\x \‘^. where / is an arb itra ry function, |x| is the size of the input x to the algorithm , fc is an aspect of the problem, and c is a constant independent of |r | and k. Given such algorithms, it may still be possible to obtain optim al solutions for large instances of A fP-hard problems for which the appropriate aspects are of small size or value. Questions about the existence of such algorithm s are most natu rally addressed w ithin the theory of param eterized com putational complexity developed by Downey and Fellows.
This thesis considers the m erits of a system atic param eterized complexity analysis in which results are derived relative to all subsets of a specified set of aspects of a given iV P-hard problem. This set of results defines an "intractability map" th a t shows relative to which sets of aspects algorithms whose non-polynomial time complexities are purely functions of those aspects do and do not exist for th a t problem. Such m aps are useful not only for delim iting the set of possible algorithms for an iV P-hard problem b u t also for highlighting those aspects th a t are responsible for this iVP-hardness.
These points will be illustrated by system atic param eterized complexity analyses of prob lems associated with five theories of phonological processing in n a tu ra l languages - namely. Simplified Segmental Gram m ars, finite-state transducer based rule systems, the KIAIMO system. Declarative Phonology, and O ptim ality Theory. T he aspects studied in these analy ses broadly characterize th e representations and mechanisms used by these theories. These analyses suggest th a t the com putational complexity of phonological processing depends not on such details as w hether a theory uses rules or constraints or has one, two, or many
levels of representation but rather on the s tru c tu re of the représentât ion-relations encoded in individual mechanisms and the internal s tru c tu re of the representations.
Exam iners:
Dr. M ichael R. Fellows. Supervisor (D epartm ent o f C om puter Science)
Dr. Valerie King. D epartm ental Member (D epartm ent of C om puter Science)
Dr. F rank Ruskey. Departm ental Member (D epartm en t of C om puter Science)
^"Pr^^^EÆjÇ^ylmwska-Higgins, Outside M ember (D ep artm en t of Linguistics)
C O N T E N T S
A b s tr a c t ü C o n te n ts iv L ist o f T a b le s v il L ist o f F ig u r e s ix A c k n o w le d g e m e n ts x i D e d ic a tio n x iv E p ig ra p h x v 1. I n tr o d u c tio n 1 2 . B a ck g ro u n d 92.1 P aram eterized Complexity A n aly sis... 12
2.1.1 C om putational Complexity T h e o r y ... 12
2.1.2 P aram eterized C om putational Complexity T h e o r y ... 21
2.1.3 System atic Param eterized Complexity A n a ly s is ... 33
2.2.1 W h at is P h o n o lo g y ? ... 42
2.2.2 Phonological R e p re s e n ta tio n s ... 47
2.2.3 Phonological Mechanisms as Finite-S tate A u t o m a t a ... 58
3. C o m p u ta tio n a l A n a ly se s o f P h o n o lo g ic a l T h e o r ie s 79 4. A S y s t e m a t ic P a r a m e te r iz e d C o m p le x ity A n a ly s is o f P h o n o lo g ic a l P r o c e s s in g u n d e r R u le - an d C o n str a in t-B a se d F o rm a lism s 93 4.1 Sim plified Segm ental G ram m ars ... 97
4.1.1 B a c k g ro u n d ... 97
4.1.2 A n a ly s is ... 105
4.1.3 Im p h c a tio n s ... 120
4.2 A M ost Useful Special Case: T h e B ounded DFA Intersection P r o b le m ... 130
4.3 FS T -B ased R ule S y s t e m s ... 139 4.3.1 B a c k g ro u n d ... 139 4.3.2 A n a ly s is ... 140 4.3.3 Im p h c a tio n s ... 147 4.4 T h e KIM M O S y s te m ... 153 4.4.1 B a c k g ro u n d ... 153 4.4.2 A n a ly s is ... 156 4.4.3 Im p h c a tio n s ... 162 4.5 D eclarative P h o n o lo g y ... 169 4.5.1 B a c k g ro u n d ... 169 4.5.2 A n a ly s is ... 173 4.5.3 Im p h c a tio n s ... 193
4.6 O ptim ality T h e o r y ... 201
4.6.1 B a c k g ro u n d ... 201
4.6.2 A n a ly s is ... 207
4.6.3 Im p lic a tio n s ... 223
4.7 Some F in al Thoughts on the Com putational Com plexity of Phonological Pro cessing ... 231
5. C o n clu sio n s 238
R e feren ces 242
A . P r o b le m I n d e x 255
B . T h e P a r a m e te r iz e d C o m p le x ity o f S im p lified S e g m e n ta l G ram m ars w ith
LIST OF TABLES
2.1 A System atic Parameterizeci Complexity Analysis of the Lo n g e s t COMMON
SUBSEQUENCE Problem ... 34
2.2 C haracteristics o f R epresentations of Phonological Theories Exam ined in this T h e s i s ... 55
2.3 C haracteristics o f Iterated Finite-State A utom aton O p e r a t i o n s ... 73
2.4 C haracteristics of Mechanisms of Phonological T heories Exam ined in this T h e s i s ... 75
4.1 The P arajn eterized Complexity of the S S G - En c o d e P r o b le m ... 126
4.2 The P aram eterized Complexity of the SSG-ENCODE Problem (C ont’d) . . . 127
4.3 The P aram eterized Complexity of the S S G - De c o d e P r o b le m ... 128
4.4 T he P aram eterized Complexity of the SSG-DECODE Problem (Gont’d) . . 129
4.5 The P aram eterized Complexity of the BOUNDED D F A INTERSECTION Problem l37 4.6 T he P aram eterized Complexity of the F S T - En c o d e P r o b le m ... 152
4.7 T he P aram eterized Complexity o f the F S T - De c o d e P r o b le m ... 152
4.8 T he P aram eterized Complexity of the K IM -En c o d e P r o b le m ... 168
4.9 T he Param eterized Complexity of the K IM -De c o d e P r o b le m ... 168
4.10 The P aram eterized Complexity of the D P - En c o d e P r o b l e m ... 200
4.12 T h e Param eterized Complexity of th e O T - En c o d e P r o b l e m ... 230
4.13 T h e P aram eterized Complexity o f th e O T - De c o d e P r o b l e m ... 230
4.14 Sources of Polynom ial-Tim e In trac ta b ih ty in the Encoding Decision Problem s A ssociated W ith Phonological Theories Exam ined in This T h e s i s ... 233
4.15 Sources of Polynom ial-Tim e In trac ta b ih ty in the Decoding Decision Problem s A ssociated W ith Phonological Theories Exam ined in This T h e s i s ... 233
4.16 T h e C om pu tatio n al Complexity of Search Problem s A ssociated W ith Phono logical Theories Exam ined in This T h e s i s ... 237
LIST OF FIG U R ES
2.1 A lgorithm Resource-Usage Complexity F u n c tio n s ... 16
2.2 The R elationship Between Hardness and Com pleteness Results in Com puta tional C om plexity T h e o r y 18 2.3 System atic Param eterized Complexity Analysis amd Polynom ial Time In tra c ta b ih ty M a p s 39 2.4 Phonological R e p r e s e n ta tio n s ... 51
2.5 Phonological R epresentations (C ont’d ) ... 52
2.6 Form D ecom position of Phonological R ep resen tatio n s... 54
2.7 A Sim plified Autosegm ental R e p re s e n ta tio n ... 57
2.8 A Sim plified Autosegm ental R epresentation (C o nt’d ) ... 58
2.9 A F in ite-S ta te A c c e p t o r ... 60
2.10 A F in ite-S ta te T r a n s d u c e r... 63
2.11 O p eratio n of a C ontextual D eterm inistic F in ite-S tate A u to m a to n ... 78
4.1 R eductions in C h ap ter 4 ... 96
4.2 Segm ental R ew riting Rules ... 99
4.3 A Subsequence D eterm inistic Finite-State A c c e p to r... 132
4.4 The D ecoding Tree C onstruction from Lemma 4.2.3 133
4.6 The KIMMO S y s te m ... 154
4.7 Full Form G ra p h s ... 181
4.8 Full Form G raphs (C o n t’d) ... 182
4.9 A Systolic D eterm inistic Finite-State A c c e p t o r ... 187
4.10 Evaluation o f C andidate Full Forms in O ptimality T h e o r y ... 203
4.11 Evaluation of C andidate FuU Forms in Optim ality T h e o ry (C ont’d ) . 204 4.12 Phonological Theories as Compositions of R ep resen tatio n -R elatio n s... 235
A cknow ledgm ents
T h is dissertation is the p ro d u c t o f several years work carried out a t the University of V ictoria an d M cM aster University. I n th a t time, I have come to owe a great deal to a great many. Lack of space precludes m entioning them all. The efforts of those not nam ed below m ust be acknowledged by the fact th a t I am, a t long last, w riting these acknowledgments.
F irst off, I would like to th a n k various departm ental staff members at the U niversity o f V ictoria and M cM aster University, namely, M arg Belec, Isabel Campos, Nancy C han, H elen G raham , Sharon M oulson, M arla Serfling, and N atasha Swain, for their courteous a n d ever-helpful escorting th ro u g h th e red ta p e th a t is an unavoidable part of b o th grad u ate studies a n d academia. I w ould also like to th an k the various system s personnel I’ve d ealt w ith, nam ely Chris Bryce, A lan Idler, WiU Kastelic, M att Kelly, and Derek Lipiec, for answ ering an almost never-ending stream o f odd questions ab o u t obscure p r o g r a m s a n d LaTeX typesetting and, th ro u g h th eir efforts, keeping my files an d my sanity intact.
I have had the good fo rtu n e to a tte n d many m eetings over the course of my P h.D ., w hich has very much enriched my work - thanks for this goes to my Ph.D. advisor, Mike Fellows, and my postdoc supervisor, Tao Jiang. I would also like to acknowledge Eric R istad for providing an N SF grant for me to atten d the DIMACS Workshop on H um an Language a t Princeton in M arch of 1992, which awakened my interest in my dissertatio n topic, an d M ark Kas for providing a very generous stu d en t g ran t for me to a tte n d th e Sum m er School in B ehavioral an d Cognitive Neurosciences a t B CN Groningen in Ju ly of 1996, which introduced m e to th e wide world of finite-state n a tu ra l language processing.
I have had the good fortune to m eet and interact w ith many colleagues while researching an d w riting m y d issertation. I would hke to th an k th e members of the T heory G roup a t University of V ictoria, nam ely Jo h n EUis, Val King, W endy Myrvold, and F rank
Ruskey, for th e ir m any calm ing talks and advice a b o u t academic life. I would also like to thank John Colem an, Jason Eisner, M ark Ellison, Susan Fitzgerald, Ayse K aram an, Lauri K arttunen, M artin Kay, A ndras K ornai, Eric Laporte, M ehryar M ohri, T hom as Ngo, Alexis M anaster R am er, Eric R istad, Giorgio Satta, Bruce Tesar, Alain T h eriau lt, Jo h n Tsotsos, G ertjan van N oord, a n d M arkus W alther for sharing unpublished m anuscripts, reprints, and dehghtful conversations over the years, both in person and by e-mail. Last but by no means least in this list are various members of th e param eterized com plexity research community, nam ely Lim ing Cai, K evin Cattell, Marco C esati, Mike Dinneen, R od Downey, P atricia Evans, Mike Fellows, an d Mike HaUett. M any of the ideas in my dissertation came from conversations w ith these people - in p articu lar, Mike HaUett, th ro u g h his early concern w ith using param eterized complexity to analyze real-world d a ta sets, is the m an behind system atic param eterized complexity analysis. To aU of them , for b o th intellectual an d em otional su p p o rt, I owe a great thanks.
I would like to th an k th e members of m y P h.D . com m ittee, nam ely Ewa Czaykowska-Higgins, Val King, Frank Ruskey, and B ruce W atson, for seeing me through th e process of w riting and and defending my dissertation. I would especiaUy hke to thank Ewa Czaykowska-Higgins, for constant encouragement, detailed and p ro m p t commentaries on various d rafts of my dissertation, and probing questions ab o ut th e relationship between com plexity-theoretic results an d linguistic research. She has m ade me re-think m any things and, in th a t process, has had a great impact on b o th my thought an d my dissertation. If I am ever called u pon to supervise graduate students, I hope th a t I can do so w ith some p a rt of the grace, energy, and intelligence she has shown in her dealings w ith me.
I would like to th a n k my friends past and present for helping me th ro u g h the bad times and m aking th e good tim es b e tte r. A short list would include M arg Belec, K athy Beveridge, G ord Brown, B ette B ultena, Kevin CatteU, Claudio Costi, BiU Day, G ianluca DeUa Vedova, Mike Dinneen, P atricia Evans, Susan Fitzgerald, Em m anuel Francois, Alan Goulding, Mike HaUett, Mike Hu, Lou Ibarra, M att KeUy, Scott Lausch, D ianne MiUer, Jo h n Nakamura, Joe Sawada, B rian Shea, BiU ThreUaU, Chris TrendaU, Jeff Webb, and X ian Zhang; th ere are m any others. T hank you aU.
I would like to th a n k my P h.D . advisor, Mike FeUows, for taking me on as his student an d helping m e to appreciate more fuUy th e beauty a t th e h eart of com p u tatio n al complexity
theory. I have adm ired his intelligence, his curiosity, his energy, his com m itm ent to rigor, and his openness a n d generosity w ith ideas since the day we m et, and will hold these qualities as ideals in my own academic life in years to come. I adm ire th e same things in my postdoc supervisor, Tao Jiang; in addition, I must thank him for his patience in allowing me to complete the research and w riteup of my dissertation a t M cM aster University.
I would like to th an k the members of my family - my parents, my brother and sister and th eir spouses, an d my nieces - for being who they are, and m aking the life th a t surrounds my work worthwhile.
Finally, I would like to th an k V it Bubenik and A leksandra Steinbergs. Many years ago. Dr. B ubenik ta u g h t me my first courses in Unguistic analysis and encouraged me to continue in linguistics; Dr. Steinbergs subsequently taught me phonology, in two of the best courses I have ever had the pleasure to take, and was my supervisor for an uncom pleted u ndergraduate honours dissertation on word stress in Russian. I owe them b oth for intro ducing me to phonology and for showing me its often unappreciated beauty. I hope th a t this dissertation is p a rtia l paym ent on th a t debt.
To Bill Day and Bill ThreUall for getting me here.
All this is fact. Fact explains nothing.
O n th e contrary, it is fact th at requires explanation.
Introduction
It is not unreasonable to assum e th a t if one is given a com putational problem , one wants th e algorithm th a t will solve th a t problem using the least am ount of com puter time. T h e discipline of algorithm design responds to this need by creating efficient algorithm s for problem s, where efficiency is usually judged in terms of ( a s y m p to t ic w o rs t-c a s e ) t i m e c o m p le x ity i.e., a function th a t upper-bounds the worst-case tim e requirem ents of an algorithm over all input sizes. As one is typically interested in larger a n d larger problem sizes (a tren d some have characterized as “living in asym ptopia” ), one would like algorithms whose worst-case time complexities grows slowly as a function of input size. Ideally, this function should be a (low-order) polynom ial. A problem th a t has such an algorithm is said to be p o ly n o m ia l t im e t r a c t a b l e and one th at does not is said to be p o ly n o m ia l t i m e in tr a c ta b le .
T here are a num ber of co m p u tatio n al problems th a t have defied alm ost fifty years of effort to create polynomial tim e algorithm s for them, and are thus widely suspected (b u t not proven to be) polynom ial tim e intractable. In p a rt to prevent w asting effort in try in g to solve other such problem s efficiently, various theories of com putational complexity have been developed firom the mid-1960’s onwards [GJ79, Pap94]. Essentially, each such th eo ry proposes a class C of problem s th a t is either known or strongly conjectured to p roperly include the class P o f polynom ial tim e tractable problems. By establishing th a t
as th e m ost com putationally difficult problem s in C, one can show th a t this problem does n o t have a polynom ial time algorithm unless the conjecture P ^ C is false. T h e strength of such conjectures is typically based on th e C-hardness of at least one of the suspected polynom ial tim e intractable problems alluded to above. The most famous and widely-used of these theories is th a t for iVP-completeness [GJ79].
W hen one m ust solve large instances o f a problem, a proof of iV P-hardness for th a t problem is often taken as a license to indulge in any and all manners of heuristic algo rith m s th a t give approxim ate solutions in polynomial time, e.g., random ized algorithm s, bounded-cost approxim ation schemes, sim ulated annealing. However, this reaction may be p rem atu re. A proof of iVP-hardness only establishes th at a problem does not have a polyno m ial tim e algorithm unless P = N P . An iV P-hard problem may still have non-polynom ial tim e algorithm s; moreover, some of these algorithm s may have time-complexity functions such th a t th e non-polynomial terms of these functions are purely functions of particu lar asp ects of th a t problem, i.e., the tim e com plexity of the algorithm is /(A:)|lari‘s w here / is a n a rb itra ry function, |r | is the size of a given instance x of the problem, k is some aspect o f th a t problem , and c is a constant independent of |z| and k. Given such algorithm s, it m ay still be possible to obtain optim al solutions for large instances of IV P-hard problems encountered in practice for which the appropriate aspects are of bounded size or value (as such bounded aspects may reduce the non-polynomial terms in the time com plexity fu n ctio n of such an algorithm to polynomials o r constants and thus make the algorithm run in polynom ial tim e).
T his raises the following questions:
• How does one determ ine if a problem o f interest has such algorithms?
• R elative to which aspects of th a t problem do such algorithms exist?
T h ese questions axe by and large unansw erable within classical theories of com putational com plexity like ?VP-completeness because these theories can show only w hether a problem does or (probably) does not have a polynom ial tim e algorithm. However, such issues can be
DF99]. T h a t is, given a problem th a t is suspected to be polynomial tim e in trac ta b le and a set of aspects of th a t problem, one can prove w ithin this theory w hether th e re does or does n ot (m odulo various conjectures) exist an algorithm for th a t problem whose non-polynom ial tim e com plexity is purely a function of those aspects.
Individual param eterized results have proven useful in answering a l g o r i t h m i c ques tions a b o u t problem s draw n from areas as diverse as VLSI design, c o m p u tatio n al biol ogy, a n d scheduling (see [DF99] a n d references). For example, one such re su lt estabUshes th a t th ere is probably no algorithm for the 3-D robot motion planning problem whose non-polynom ial tim e complexity is purely a function of the num ber of jo in ts in the given ro b o t [CW95]. In this thesis, I will discuss the m erits of a system atic p aram eterized complex ity analysis in which param eterized results are derived relative to all su b sets of a specified set of aspects of a given iV P-hard problem . Modulo various conjectures, th is set of results defines a p o ly n o m ia l tim e i n t r a c t a b i l i t y m a p that shows relative to which sets of aspects non-polynom ial algorithm s do and do not exist for th a t problem . Such maps are useful not only for delim iting th e set of possible algorithms for a p a rtic u la r iV P-hard problem b u t also for h i g h l i g h t i n g those aspects of th a t problem th a t are responsible for this iV P-hardness, i.e., those aspects o f th a t problem that are s o u rc e s o f p o ly n o m ia l- tim e i n t r a c t a b i l i t y .
T h e process of creating an d using intractability maps will be illu strated by system atic param eterized complexity analyses o f problems associated w ith five theories o f phonological processing. Phonology is the area o f linguistics concerned w ith the relatio nship between spoken (surface) and m ental (lexical) forms in natural languages [Ken94]. E ach theory of phonological processing proposes types of representations for lexical and surface forms and m echanism s th a t implement th e m appings betw een these forms. Two problem s associated w ith each such theory are the encoding an d decoding problems, which are concerned with th e operations w ithin th a t theory of creating surface forms from lexical forms a n d recovering lexical forms from surface forms, respectively. T he following five theories o f phonological processing will be exam ined in this thesis;
2. F in ite-sta te transducer (FST) based rule system s [KK94].
3. T h e KIM MO system [Kax83, Kos83].
4. D eclarative Phonology [Sco92, SCB96].
5. O p tim ality Theory [MP93, PS93].
These theories together co n stitu te a historical and methodological continuum of the various types o f theories th a t have been used over th e last 30 years to describe phonological phenom ena. T he encoding an d decoding problems for each theory will be analyzed relative to a set of aspects th a t broadly characterizes both the representations and mech anism s proposed by th a t theory. In addition to providing some of th e first iVP-haxdness results for problem s associated w ith several of these theories, these analyses also comprise th e first form al framework in which various published speculations ab ou t the sources of polynom ial-tim e intractabihty in these theories can be investigated.
G ra n d rhetoric aside, a note is in order concerning the choice of th e five theories fisted above for analysis in this thesis. O ptim ality Theory, D eclarative Phonology, and F S T -based rule systems are obvious candidates for analysis, as O p tim ality Theory and D eclarative Phonology are th e m ost popular descriptive frameworks a t the present tim e and th e finite-state methods underlying FST-based rule systems are the preferred m an n er of software im plem entation of such frameworks. The cases for Simplified Segmental G ram m ars an d th e KIMMO system are slightly weaker, as b o th are older formalisms that are no longer in favor. However, they are still well worth analyzing, b o th because they are two of th e very few linguistic theories th a t have had their com p u tatio n al complexities analyzed a n d debated in the lite ra tu re and, perhaps more im portantly, because th eir com p u ta tio n a l mechanisms are much m ore closely related t r those in currently-favored theories th a n m any researchers seem to realize, cf. [Kar98], a n d analyses of these older theories make for in terestin g comparisons w ith results derived for the more current theories.
System atic param eterized complexity analysis is defined in Section 2.1.3. T his section also contains a discussion of w hat aspects of a problem can be usefully considered to be responsible for the polynomial-time intractability of a com putational problem, and gives the first formal definition of a source of polynomial-time intractability.
A new type of finite-state autom aton called a contextual finite-state auto m ato n is defined in Section 2.2.3 to assist com plexity-theoretic analyses of th e role of bounded constraint context-size in phonological processing.
C h ap ter 4 gives the first system atic param eterized com plexity analyses for Simplified Segmental Grammars, FST -based rule systems, the KIM MO system. D eclarative Phonology, and O ptim ality Theory. As such, it extends and provides th e first complete proofs of various results th a t were given (often w ithout proof) in previously pubhshed param eterized analyses of Simplified Segmental G ram m ars [DFK-f-94] and Declarative Phonology and O ptim ality Theory [War96a, War96b]. T hese analyses are particularly notable in th a t they are the first to address the role of constraint context-size in the com putational complexity of phonological processing. T he results derived in the analyses described above are used to refute several conjectures made in the literature concerning the sources of polynom ial-tim e in tractab ility in Simplified Segmental G ram m ars, FST-based rule systems, and the KIM M O system. As the param etric reductions used in these analyses are also polynom ial-tim e many-one reductions, the following results are also obtained:
— Section 4.1.3 gives the first iVP-haxdness proof for the Fi n i t e-STATE TRANSDUCER COMPOSITION problem and proofs in Section 4.3.2 extend this result to the restricted case of e-free FST composition. These results in tu rn give th e first iVP-haxdness proofs for the encoding and decoding problem s associated w ith FST-based rule systems.
— Section 4.5.2 gives the first IV P-hardness proofs for the encoding and decoding problems associated with D eclarative Phonology.
problems associated w ith a form ulation of O ptim ality Theory th a t is simpler th a n th a t proposed in [Eis97a]. T hese proofs in tu rn are used to give the first iV P-hardness proof for the problem of learning constraint-rankings in O ptim ality Theory when surface forms in stead of full forms are given as exam ples [Tes96, Tes97a, Tes97b].
T he results in Sections 4.3.2 and 4.4.2 c a n also be interpreted as the first param eterized analyses of the e-free FST composition a n d intersection operations w ithin th e frame work of Kay and K aplan’s finite-state calculus ([KK94]; see also [KCGS96, XFST]).
As is discussed in more detail in Section 4.7, the results described above suggest th a t the com putational complexity of phonological processing depends not on such details of theory form ulation as w hether a theory uses rules or constraints or has one, two, or many levels of representation but rath er on the s tru c tu re of the representation-relations encoded in individual mechanisms and the internal stru c tu re of the representations used within th a t theory.
U ltimately, the main contributions of th is thesis are the results derived in th e system atic param eterized complexity analyses described above and the (hopefully, linguistically relevant) conclusions drawn from these results. T he techniques used are by no m eans new - polynom ial-tim e intractability maps have previously been constructed for various problems [BDFW95, BDF-f95, Ces96, CW95, Eva98, HaI96, War96a] and many com plexity-theoretic (albeit unparam eterized) analyses of linguistic theories have been done over th e last 25 years (see [BBR87, Ris93a] and references). W h a t is different here is the size of th e derived polynom ial-tim e intractability maps and the use of such maps to compare a n d contrast several closely-related real-world com putational problems, cf. the extensive param eterized analysis of Turing machine com putation given in [Ces96j. The intractability m aps given in this thesis are adm ittedly incomplete in th a t all possible results have not been derived relative to th e sets of aspects considered here, there are other aspects of the problem s th a t are not considered here at ail, and no a tte m p t has been made to derive the b est possible non-polynom ial algorithms where such algorithm s have been shown to exist. However, this
complexity analyses should be done and interpreted.
The intended audience of this thesis can be split into three groups:
1. C om putational complexity theorists (ideally those who are disillusioned by th e in creasing abstractness of the field an d would like to know where it contacts reality).
2. A lgorithm designers (ideally those who wonder if those in the first group have done or ever wfil do anything th a t is relevant to them ).
3. C om putational linguists (ideally those who wonder if any com puter scientists (particularly those in the first two groups) have done or ever will do anything th a t is relevant to them ).
M y hope is th a t system atic param eterized complexity analysis will form at least two kinds of bridges between these groups: a “little bridge” betw een com putational com plexity theorists and algorithm designers (by showing how cutting-edge com plexity-theoretic tech niques can b o th be in itiated by and apphed to practical problem s in algorithm design) and a “big bridge” betw een com puter scientists and com putational hnguists (by showing how techniques from com puter science can be used both to derive b e tte r com puter im plem en tatio n s for problem s from hnguistics and to suggest poten tially useful restrictions on the com putational power of the linguistic theories associated w ith these problems).
This thesis is organized as follows. C h ap ter 2, B a c k g r o u n d , consists of two sections which give overviews of param eterized com plexity analysis and phonology. T he introduc tio n to this chapter also gives general n o tatio n th a t will be used throughout this thesis. C h ap ter 3, C o m p u ta tio n a d A n a ly s e s o f P h o n o lo g ic a l T h e o rie s , ties together the m aterial presented in C hapter 2 by describing how com putational analyses of phonolog ical theories are done in practice and showing how certain flaws in such analyses th a t are introduced by using classical theories of com putational com plexity such as iVP-completeness can be rem edied by applying the conceptual framework a n d techniques of param eterized com plexity analysis. C h ap ter 4, A S y s te m a tic P a r a m e t e r i z e d C o m p le x ity A n a ly s is o f P h o n o lo g ic a l P r o c e s s in g in R u le - a n d C o n s tr a i n t- B a s e d F o rm a lis m s , devotes a
section to the system atic param eterized com plexity analysis of each of the five phonological theories m entioned above. Each section is broken into subsections which give an overview of a p articu lar theory, the analysis of th a t theory, and a discussion of the im plications of the results derived w ithin th a t analysis an d suggestions for future research. The organization of these discussions is somewhat involved and is described in more detail in th e introduction to this chapter. This chapter concludes w ith a brief discussion of the im plications of all results derived in this thesis for phonological processing in general. C h ap ter 5, C o n c lu s io n s , summarizes the m ain contributions of this thesis and gives directions for future research common to ail theories exam ined in C h apter 4. T he thesis finishes up w ith two appendixes, the first giving the pages where th e com putational problem s used in this thesis are first defined, an d the la tte r giving the first pubhshed proofs of results for Simplified Segmental G ram m ars th a t were given w ithout proof in [DFK+94].
Background
This ch ap ter is divided into two m ain sections, each of which gives an introductory overview of a topic addressed within this thesis. T h e first section is on param eterized complexity analysis, an d consists of reviews of com putational complexity theory an d param eterized com putational complexity theory and a discussion of how param eterized complexity anal yses can be applied in a system atic m anner to determ ine the sources of polynom ial tim e in tractab ility in com putational problem s. T he second section is on phonology, and gives a b rief introduction to the phenom ena stu d ied w ithin phonology as well as the various types o f representations and mechanisms by which these phenomena are described w ithin phonological theories.
T he following notation will be used throughout this thesis. The m ain objects of interest will be sets of objects, strings of symbols over some alphabet, and graphs. Some operators wiU have shghtly different meanings depending on w hether th e operand is a set or a string.
• S e ts o f O b je c ts : A s e t is a collection of objects. W ith respect to a set S , let l^l denote the num ber of elements in S , i.e., the size of S, and 0 denote th e em pty set, i.e., th e set w ith no elements. Given a set A, let { x i,X2, . . . } denote th e elements of
A and x E A denote th a t a: is an elem ent of A . Given two sets A and 5 , let A U R be th e union of A and B , i.e., the set consisting of the elements in A a n d /o r B , A x B
be the C artesian product of A an d B , i.e., the set consisting o f all ordered pairs (a:, y) such th a t X E A and y E B , and A — B he the difference o f A and B , i.e., the set consisting of all elem ents in A th a t are not also in B .
• S tr in g s o f S y m b o ls: A s t r i n g is a sequence of symbols d raw n from some alphabet. W ith respect to a string x over an alphabet E, let |a:| denote the num ber of symbols in X, i.e., the le n g th of x, and e denote the em pty string, i.e., the string of length 0. Given a string x, let xiX2 • • • X|^|, Xj E S for 1 < i < |a;|, denote the concatenation of symbols from S th a t is x, and x y = xqx2 . . . x\^\yi.y2 ■ - • y\y\-, Xj G E for 1 < i < |x| and
yi E E for 1 < i < |y|, be the string formed by concatenating the symbols in string y onto the right end of strin g x. Given a string x = xiX2 . . . x„ and an integer k < n, the fc-length p refix o f x is the string x i, X2 . . . Xfc and the A:-length sufHx o f x is the string Xji_(fc_i)X„_(jt_2) • - -^n- Let E ", E - " , E*, and S"^ denote th e sets of all strings over E th a t have length n, length less th an or equal to n, length greater than or equal to zero, and length strictly greater than zero, respectively. A (fo rm a l) la n g u a g e over an alphabet E i s a subset of E*. Occasionally, it will be useful to encode a set
k
of strings into a single strin g w ithout specifying the details; let () : x E* i-> E* be an invertible function th a t encodes a set of k given strings over E onto a single string over E.
• G ra p h s : A g r a p h G = {V. E ) is a set of v e rtic e s V and a set of e d g e s E th a t Unk pairs of vertices, i.e., E Ç .V y .V . lia-n edge between vertices u an d v has a direction from u to V, the edge is called a d ir e c t e d e d g e or a rc and is w ritten (u, u); otherwise, if the edge has no direction attached, it is called an u n d i r e c t e d e d g e and is w ritten (u .v ). If all edges in a graph G are directed, it is called a d i r e c t e d g r a p h and may alternatively be w ritten as G = {V .A ). If all edges in G are undirected, G is an u n d ir e c t e d g ra p h . Unless otherwise noted, all graphs in th is thesis are undirected. If two vertices u and u in G are joined by an edge (if G is undirected) or there is an arc from u to v (if G is directed), then u is a d ja c e n t to v in G; otherwise, u and v are not adjacent in G.
A final note is perhaps in order for readers who are not linguists. O ne of the most infiuential works in m odern phonology is Chomsky a n d H alle’s The Sound P attern o f English [CH68]. Following common practice in the hnguistics literatu re, the acronym S P E will be used th ro u g ho u t this thesis to denote either th e phonological theory or phonological rule-system s of th e type described in [CH68].
2 .1
P a r a m e te r iz e d C o m p le x ity A n a ly sis
2 .1 .1 C o m p u t a t i o n a l C o m p l e x it y T h e o r y
T h e two basic concepts of interest in com putational complexity theory are problem s and algorithm s. Essentially, a problem is a question and an algorithm is a finite sequence of in stru ctio n s for some com puter which answers th a t question. C om putational complex ity th eo ry establishes up p er and lower bounds on how efficiently problems can be solved by algorithm s, where “efficiency” is jud g ed in terms of the am ounts of com putational resources, e.g., tim e or space, required by a n algorithm to solve its associated problem. As n o ted by Rounds [Rou91, page 10], “T h e presuppositions of an already established theory, such as complexity theory, are perhaps the properties of the theory m ost easily ignored in m aking an apphcation” . Yet, it is precisely these presuppositions th a t we need to be aw are o f to hilly appreciate b o th th e advantages of param eterized com plexity analysis and how such analyses can improve on previous com putational analyses of phonological theories. W ith this in m ind, the basics of com putational complexity theory are reviewed in this section. T h e m aterial given below is draw n largely from Sections 1.2, 2.1, an d 5.1 of [GJ79]. R eaders w ishing more detailed treatm ents are referred to [GJ79, HU79, Pap94j.
Let us first look a t com putational problem s. A c o m p u ta tio n a l p r o b le m is a question to b e answered, typically containing several variables whose values are unspecified. A n in s ta n c e of a problem is created by specifying particu lar values for its variables. A problem is described by specifying b o th its instances and the n a tu re of solu tions for those instances. Problem s typically have two levels of description - an abstract description in term s of the general stru ctu re an d type of instances and solutions, and a concrete description in term s of the form al objects, e.g., languages and strin g relations, onto which instances and solutions are m apped for m anipulation by com puter models and subsequent analysis. There are m any types o f problems. Two of the most com m on types are described below.
D e f in itio n 2 .1 .1 [GJ79, Section 2.1] A d e c is io n p r o b le m II is a set D u o f instances and a set Y u Ç D u o f yes-instances. A decision problem is described informally by specifying:
1. A generic instance in terms o f its variables.
2. A yes-no question stated in terms of the generic instance.
A decision problem II is form ally described by a language Zr[II, e] Ç S ’’" fo r some alphabet S , where e : Yn i—)■ S"*" is an instance encoding fu n ctio n and L = {a: | 3 / G Kn such that e{I) - x } .
D e fin itio n 2.1 . 2 [GJ79, Section 5.1] A s e a rc h p r o b le m II is a set D^i of instances and
a set Su o f solutions such that fo r each I 6 Du- there is an associated set S'n[-I] Q S u o f
solutions fo r I . A search problem is described inform ally by specifying: 1. A generic instance in terms of its variables.
2. The properties that m ust be satisfied by any set o f solutions associated with an instance created from the generic instance.
A search problem II is form ally described by a string relation -R[II, e] Ç S ’*" x fo r some alphabet S , where e = {eD .es), ep ■ Du and es ■ S u E"*", is a pair of instance and solution encoding functions and i2[II, e] = { { x ,y ) \ 31 E Yu 3 8 G S'nM such that a n il) = a: and e s{ S ) = y }.
T he instance and solution encoding functions should be “reasonable” in the sense th at they are concise and decodable - th a t is, they create encodings th a t are not artificially larger th an the inform ation in the given instance w arrants and th a t can be decoded easily to extract any of th e variables specified in the generic instance [GJ79, p. 2 1].
E x a m p le 2 .1 .3 C onsider the problem of finding vertex covers associated with a given graph, i.e., a subset of the vertices of a a given graph such th a t each edge in the graph has a t least one endpoint in the subset. Two possible informal descriptions of decision and search versions of th is problem are given below:
Ve r t e x c o v e r (Decision) [GJ79, Problem G T l]
Instance: A g rap h G = (V, £?), a positive integer k.
Question: Is th ere a vertex cover of G of size a t m ost k, i.e., a set of vertices V Ç V, \V'\ < fc, such th a t for every edge {x, y) 6 E , either a: or y is in V'7
Ve r t e x c o v e r (Search)
Instance: A g ra p h G = {V ,E ), a positive integer k. Solution: Any vertex cover of G of size < k.
In form al descriptions of theses problem s, the input graphs could be specified as \V\ x \V\ edge-adjacency m atrices and instances could be encoded as strings of the form ([V |,G , A:) w ith length logg \V\ + \V \‘^ + log2 k b its. In the case of the search problem, if a num bering from I to IV| on th e vertices is assum ed, each solution is a subset of ( 1 , . . . , |V|} and can
be represented by a strin g of |V| bits.
I
Inform al descriptions are useful to th e extent th a t they are faithful to our m ental intuitions a b o u t w hat problem s are and how th ey behave, and formal descriptions are useful to the extent th a t they b o th m irror informal descriptions and are am enable to m athem atical an al ysis. In short, formal descriptions m ake reasoning abo u t problem s rigorous and inform al descriptions ensure th a t such reasoning rem ains relevant; hence, each has a role to play in com putational com plexity theory.
Given th e above, an a lg o r it h m A for a problem II is a finite sequence of in stru c tions for some com puter which s o lv e s II, i.e., for any given instance of II, A com putes th e ap p ro p riate solution. W hat co n stitu tes an appropriate solution depends on the type of the problem . For example, the solution to a decision problem is “Yes” if the given instance is in Yri and “No” otherwise, an d the solution to a search problem is “No” if th e solution-set associated w ith the given instance is em pty an d a n arb itra ry element o f this solution-set otherw ise. Note th a t as we are now discussing problem s and algorithm s in relation to form al com puter models, the instances an d solutions m anipulated by algorithm s are th e string-encodings o f th e ab stract instances an d solutions discussed above.
Readers interested in the details o f how problems and algorithm s are m apped onto formal com puter m odels are referred to C hapters 2 and 5 of [GJ79].
We can now talk ab o u t algorithm efficiency. Typical com putational resources of interest are tim e and space, which correspond to the num ber of instructions executed or the am ount of m em ory used by th e algorithm when it is im plem ented on some sta n d a rd type o f com puter, e.g., a determ inistic Turing machine (for detailed descriptions o f the various kinds of T uring machines, see [GJ79, HU79, LP81]). For some resource R and problem II, let Ra. : D ll I—>• A/" be be the function th a t gives the am ount of resoirrce R th a t is used by algorithm A to solve a given instance of II. T he resource-usage behavior of an algorithm over all possible instances of its associated problem is typically stated in term s of a function o f instance size th a t siunm arizes this behavior in some useful marmer. T h e creation of such functions has th ree steps:
1. Define an instance-length function such th a t each instance of the problem of interest can be assigned a positive integer size. Let the size of instance I be denoted by |/ |.
2. Define a “raw ” resource-usage function th a t summarizes the resource-usage behavior of A for each possible instance size. Let — {R.4(7) \ I is an instance of th e problem solved by A an d | / | = n} be the R-requirements of algorithm A for all instances of size n . For each instance-size n, choose either one elem ent of or some function of R \ to represent 72^. Several p o p ular ways of doing this are:
• T h e highest value in R \ (w orst-ca se). • T he lowest value in RJ\ (b e s t-c a s e ).
• T h e average value of 72^ relative to some probabihtj’^ distribution on instances o f size n (a v e r a g e-c a se).
Let Sa ■ A7 be the function th a t gives this chosen value for n > 0.
3. “Sm ooth” the raw resource-usage function 5.4(n) via a function Ca ■ A7i->- A7 th at asym ptotically bounds 5 a(ri) in some fashion. Several standard types o f asym ptotic bounding functions g on a function / are:
C S S
Instance Size
Figure 2.1: A lgorithm Resource-Usage Com plexity Functions. Let A be the given algorithm for a problem II w ith instance-set D[%. In the figure above, each of the sm all circles denote the resource-usage value Ra{^) of A for a particu lar instance I G D u, the large circles denote the largest resource-usage value in for each instance size n, the do tted fine denotes the “raw ” worst-case resource-usage function S',4(n), and the sofid line denotes th e asym ptotic upper bou n d worst-case resource-usage function 0 4 (^1) which “smoothes” S'^(n). See m ain text for fu rth er explanation of terms.
• A s y m p to tic u p p e r bound: / G 0 {g ) if there exists a constant c a n d no > 0 such th a t for all n > no, / ( n ) < c • g in ).
• A s y m p to tic low er bound: / G if there exists a constant c a n d no > 0
such th a t for all n > no, / ( n ) > c ■ g in ).
• A s y m p to tic tig h t bound: / G 0 (g ) if / G 0 (g ) and / G Dig).
The function C Ain) is called a R -c o m p le x ity f u n c tio n for algorithm A . T h e process of creating such functions is illustrated in Figure 2.1. The complexity functions m ost com m only used in the literatu re (and hence in th is thesis) choose resource R to be tim e relative to a determ inistic T uring machine an d sum m arize Rx(-f) hi term s of asym ptotic upper bounds on the worst case; hence, th ey are known as asym ptotic worst-case tim e
complexity functions. Given th a t a n algorithm ’s asym ptotic w orst-case time complexity function is / ( n ) , th e algorithm is said to be a /( n ) time algorithm , a n d its associated problem is said to be solvable in f { n ) time. An algorithm is e ffic ie n t if its complex ity function satisfies some criterion, e.g., th e complexity function is a polynom ial of the instance size. A problem is t r a c t a b l e if it has an eflBcient alg o rith m ; otherwise, the problem is said to be i n tr a c ta b l e . As there are many possible c rite ria which can be used to define efficiency, there are many possible types of tra c ta b ility an d intractabiUty.
C om putational complexity theory is concerned w ith estab lish in g not only w hat problems can b e solved efficiently b u t also w hat problems cannot be solved efficiently. The form er is done by giving an efficient algorithm for the problem, an d th e la tte r by defining th e following four components:
1. A universe lA of problems.
2. A class 7” C if of tractable problems.
3. A reducibility oc between pairs of problems in U. A r e d u c tio n &om a problem II to a problem II' (w ritten II a II') is essentially an algorithm th a t can use any algorithm for n ' to solve n . A r e d u c ib ility is a set of reductions th a t satisfies certain properties. T he reducibilities of interest here m ust satisfy the following tw o properties:
(a) T r a n s itiv ity : For all problems II, II', II" € if, if II oc I I ' an d II' oc II" then
n
ocn".
(b) P r e s e r v a t io n o f T r a c ta b ility : For aU problems II, II' G Z7, if II oc II' and n ' G T th en H G T .
4. One or more classes of problem s C C.U. such th a t T CZ C.
Given a class of problems /C C Z/, a problem II is /C -hard if for aU problems II' G 1C, n ' oc II; if n is also in IC, th en II is /C -co m p lete. Informally, th e reducibility oc estab lishes the com putational difficulty o f problems relative to each o th e r - th a t is, if II oc II' th e n n ' is a t least as com putationally difficult as II. Hence, /C-complete problem s are the
C-Hard
C-Complete
Figure 2.2: The R elationship Between H ardness and C om pleteness R esults in C o m p utation al Complexity Theory. See m ain tex t for explanation o f symbols.
m ost com putationally difficult problem s in /C, an d /C-hard problem s are a t least as com p u ta tio n a lly difficult as the m ost com putationally difficult problem s in /C. T hese notions are significant because if a given problem H is C -hard relative to oc for any class C and reducibility oc as defined above th en H is not in T" and hence does not have a n efficient alg o rith m (see Figure 2.2).
In practice, it is often very difficult to prove th a t a problem is h a rd for classes C as defined above. In these cases, it is convenient to use shghtly weaJcer classes C such th a t it is strongly conjectured (but n o t proven) th a t T C C . One can still derive hardness and
completeness resu lts relative to such, classes: however, the vahdity of the conclusion th a t a C'-hard problem does not have an efficient algorithm now depends on the stren g th o f the conjecture th a t 7” 7^ C'. Though such conjecture-dependent results are weaker th a n actu al intractability resu lts, such results can function as proofs of intractabiU ty for aU practical purposes if the s tre n g th of the conjectures can be tied to practical experience in developing (or rather, faiUng to develop) efficient algorithm s for C'-hard problems.
Such has b een the case for polynom ial-tim e intractability. Polynom ial time algorithm s are useful in p ractice because the values of polynom ial functions grow much more slowly th a n the values o f non-polynom ial functions as instance size goes to infinity, and hence algorithm s th a t ru n in polynom ial tim e can solve much larger instances in a given period of tim e th a n those th a t ru n in non-polynom ial tim e (for a graphic illustration of th is, see [GJ79, Figure 1-2]). A num ber of classes of decision problem s th a t properly include th e class of decision problem s th a t are solvable in polynomial tim e were developed in the 1960’s, e.g., E X P T I M E . However, very few problem s of interest have been shown to be haxd for these classes. This m o tiv ated the developm ent of w hat is arguably the most famous th eo ry of com putational com plexity to date, namely, the theory of iVP-completeness (see [GJ79] and references). R elative to the scheme above, the com ponents of this theory are:
U: The universe o f decision problems.
T : The class P o f decision problems th a t can be solved in polynom ial tim e by an algorithm running on a determ inistic T uring machine.
oc: The polynom ial tim e many-one reducibiUty <m, i.e., given two decision problem s EE and n ', n <m n ' if there exists an algorithm th a t, given an instance x of II, co nstru cts an instance x ' of IT in tim e polynom ial in |x| such th a t x € Tn if and only if x ' G
Yn'-C: T he class N P o f decision problem s th a t can be solved in polynom ial tim e by an algorithm running o n a nondeterm inistic Turing machine. Such an algorithm essentially has access to som e polynom ial num ber of bits, and is said to solve its associated problem if the execution of th a t algorithm relative to a t least one of the possible choices of values for these bits solves the problem in a polynom ial num ber of steps.
As any determ inistic polynomial tim e algorithm can be rephrased as a nondeterm inistic polynom ial tim e algorithm th a t ignores the values of the nondeterm inistic bits, P C N P . If a decision problem can be shown to be A’P -h a rd then it is not in P (and hence does not have a polynom ial tim e algorithm) modulo th e stren g th of the conjecture th a t P ^ N P . M any theorem s (which themselves are stated relative to other conjectures) su p p o rt this conjecture (see [Sip92] an d references); however, the best evidence to date th a t P ^ N P is th a t in alm ost 50 years o f algorithm research, no one has yet produced a polynom ial tim e algorithm for a decision problem th a t is iV P-hard. Hence, it is a widely accepted working hypothesis th a t P ^ N P .
W hen interpreting iVP-hardness results, it is very im portant to remember th a t an ATP-hardness result for a problem H only implies th a t H is polynomial-time intractable if P ^ N P . However, given the widespread confidence in this working hypothesis, an iV P-hardness result can, for aU practical purposes, be considered equivalent to a proof of polynom ial-tim e intractability, and will be tre a ted as such in the rem ainder of this thesis.
T h e focus above on decision problems m ay seem irrelevant as it is search problem s th a t axe typically of interest. However, the theory of iVP-completeness (as are many theories of com p u tatio n al complexity) is phrased in term s of decision problems for two reasons:
1. T h e formal model underlying decision problem s, i.e., formal languages, is much easier to m anipulate and analyze th an the formal model underlying search problems, i.e., strin g relations.
2. As eflScient algorithm s for search problems can be used to solve appropriately-defined decision problems efficiently, intractability results for such decision problems im ply (an d can be used to prove) the intractability of their associated seaxch problems. For instance, if it can be shown th a t any polynomial tim e algorithm for a search p roblem of interest can be used to solve an appropriately defined decision problem in polynom ial tim e an d th a t this decision problem is ATP-hard (and hence not solvable in polynom ial tim e unless P = N P ) , th en the search problem cannot be solved in polynom ial tim e unless P = N P .
E x a m p le 2 .1 .4 Such a relationship holds for the vertex cover problem s defined in Exam ple 2.1.3. Any algorithm for th e search version of VERTEX COVER also solves the deci sion version of V ER TEX COVER, and as the decision version is iVP-complete (see [GJ79, Problem G T l] and references), the search version of V E R T E X c o v e r does
not have a polynom ial tim e algorithm unless P = N P . |
Ideally, a com plexity-theoretic analysis of a problem is not ju s t a one-sided quest for eith er algorithm s o r hardness results. R ather, it is an ongoing dialogue in which b o th types o f results are used to fully characterize th e problem by showing w hich restrictions make th a t problem tractab le an d which do n ’t [GJ79, Section 4.1].
2 .1 .2 P a r a m e t e r i z e d C o m p u t a t i o n a l C o m p le x it y T h e o r y
T he theory o f iVP-com pleteness described in the previous section was developed to show which problem s probably do not have polynomial time algorithms. Since th e inception of this th eo ry in 1971 w ith C ook’s proof th a t Bo o l e a n C N F FORMULA s a t i s f i a b i l i t y is
iV P-com plete [Coo71], thousands of other problems have b een shown to be ATP-hard an d ATP-complete. T hough it is nice to know th a t such problem s do not have polynom ial tim e algorithms unless P = N P , the inconvenient fact rem ains th a t these problem s (especially those w ith real-world applications) must still be solved.
How does solve iV P-hard problems in practice? To date, th e two most popular approaches w ithin com puter science have been:
(1 ) Invoke some type of non-polynom ial time “b ru te force” optim al-solution technique, e.g., dynam ic program m ing, branch-and-bound search, m ath em atical programming.
(2) Invoke some type o f polynom ial time approxim ate-solution algorithm , e.g., bounded-cost approxim ation schemes, randomized algorithm s, sim ulated annealing, heuristics.
W hen th e instances to be solved are large, approach (1) may not be feasible, thus forcing the invocation o f approach (2). Indeed, fast approxim ation algorithms seem the most popular
m eth o d for dealing w ith ATP-hard problems a t this time. However, there is a n o th er approach (which is arguably a n inform ed version of (1)):
(3 ) Invoke a non-polynom ial tim e algorithm th a t is effectively polynom ial tim e because its non-polynom ial tim e complexity is purely a function of some set of aspects of the problem th a t are of bounded size or value in instances of th a t problem encountered in practice.
Serious discussion o f this alternative requires the following definitions.
D e f in itio n 2 .1 .5 G iven a decision or search problem II, an a s p e c t a o f H is a fu n ctio n a : Dxi 1-4- 2 + fo r som e alphabet E.
D e f in itio n 2 .1 .6 G iven a decision or search problem H, aspects a and a' o f H, and a non-polynomial tim e algorithm P fo r H, t h e n o n -p o ly n o m ia l t i m e c o m p le x ity o f P is p u r e l y a f u n c tio n o f a i f the tim e complexity function o f P can be w ritten as f{a)\a'\'^, where f : E"^ t-f f f is an arbitrary function and c is a constant independent o f a and a '. A n aspect of a problem is essentially some characteristic th a t can be derived from instances of th a t problem. Convenient aspects of a problem are th e variables in a generic instance o f th a t problem or various num erical characteristics of those variables. For example, some aspects of the vertex cover problem s defined in Exam ple 2.1.3 are th e given g raph (G ), the n um ber of vertices in th a t g rap h (IVj), the given bound on th e size of the vertex cover (A:), or th e m axim um degree of any vertex in G . Each problem has m any aspects, and some problem s have aspects of bounded size or value th a t are potentially useful in the sense of (3) above.
E x a m p le 2 .1 .7 Consider the following decision problem from robotics:
3-D Ro b o t m o t i o n p l a n n i n g
Instance'. A robot composed of linked polyhedra, an e n v ir o n m e n t composed of some set o f poly h ed ral obstacles, and initial and final positions p i and p f of the robot w ithin th e environm ent.
Question: Is there a sequence of motions of the robot, i.e., a sequence of translations a n d ro ta tio n s of th e finked polyhedra making up the robot, th a t move it from p / to p p w ithout intersecting any of th e obstacles?
T h is problem is known to be P S P A C E -h a x d [Rei87] an d hence does not have a polynom ial tim e algorithm unless P = P S P A C E . T hough certain aspects such as the num ber of or description-com plexity of the environmental obstacles tend to be very large, other aspects do n ot, e.g., th e num ber of joints in the robot is less th a n 7 for commercially-available ro b ot arm s an d less th a n 20 for robot “hands” (see [CW95] and references). An algorithm for this problem whose non-polynom ial time com plexity is purely a function of the num ber of jo in ts of th e given robot m ight be useful in practice. |
T h e re seem to be m any other real-world problems th a t have such bounded aspects [BDF-f-95, DFS98], an d anecdotal evidence suggests th a t non-polynom ial tim e algorithms th a t exploit these aspects play an im portant role in solving iV P -hard problems th a t occur in commercial a n d in d u stria l applications [DFS98].
T h e attractiveness of non-polynomial tim e algorithm s th a t exploit aspects of bounded size or value im m ediately suggests two questions:
(1 ) How does one determ ine if a problem of interest has such “reasonable” non-polynomial tim e algorithm s?
(2) R elative to which aspects of th a t problem do such algorithm s exist?
(3) G iven a problem and a set of aspects of th a t problem, does there exist an algorithm for th a t problem whose non-polynom ial time com plexity is purely a function o f those aspects?
This question can be partially addressed w ith in classical theories of com putational com plexity like th a t for jVf-completeness. One can try using conventional algorithm -design strategies to find such an algorithm. Alternatively, one can show th a t such an algorithm does n o t exist by establishing the polynomial-time intractability of the version of th a t problem in which the values of those aspects are fixed (in the case of numerical aspects, to sm all constants).
E x a m p le 2 .1 .8 Consider the 3-D robot m otion planning problem defined in Example 2.1.7. If one could establish the PSPACE-h.axdn.ess of the version of this problem in which the n u m b er of joints in the given robot is fixed a t some constant c, then the 3-D robot m otion planning problem would not have an a l g o r i t hm whose non-polynomial tune com plexity is purely a function of the num ber of joints in the robot unless P = P S P A C E (as th e polynom ial-tim e algorithm created by fixing the number o f joints to c in any such non-polynom ial tim e algorithm could be used in conjunction w ith th e P S 'P A C E -h ard n ess reduction for the 3-D robot motion p la n n in g problem to solve any problem in P S P A C E in
polynom ial tim e). |
Such proofs axe p a rt of the strategy advocated in G arey and Jo h n so n ’s discussion on how to use algorithm s and IVP-hardness results to m ap a problem ’s “firontier of tractab ility ” [GJ79, Sections 4.1 and 4.3]. However, it is not obvious th a t one can, for every possible aspect o f a problem , always either give the required type of non-polynom ial time algorithm relative to th a t aspect or establish the polynom ial-tim e intractability of th a t problem rela tive to some fixed value of th a t aspect. A large p a rt of the diflhculty here stems firom one of the idealizations underlying classical theories of com putational com plexity - namely, th a t instances are indivisible and are characterized in analyses by a single instance size value. As long as it is impossible to extract aspects of an instance and consider their effects in isola tion on th e tim e complexity of an algorithm for a problem , attem p ts to analyze the effects of
