Table of Contents

Sections marked with * are optional, in the sense that later material does not depend on them.


PREFACE xiii
ACKNOWLEDGEMENTS xvii
CREDITS xix
PART I: INTRODUCTION 1
1Why Study the Theory of Computation?2
1.1The Shelf Life of Programming Tools2
1.2Applications of the Theory Are Everywhere 5
2Languages and Strings8
2.1Strings 8
2.2 Languages 10
Exercises 19
3The Big Picture: A Language Hierarchy21
3.1 Defining the Task: Language Recognition 21
3.2 The Power of Encoding 22
3.3 A Machine-Based Hierarchy of Language Classes 28
3.4 A Tractability Hierarchy of Language Classes 34
Exercises 34
4Computation36
4.1 Decision Procedures 36
4.2 Determinism and Nondeterminism 41
4.3 Functions on Languages and Programs 48
Exercises 52
PART II: FINITE STATE MACHINES AND REGULAR LANGUAGES 53
5Finite State Machines54
5.1 Deterministic Finite State Machines 56
5.2 The Regular Languages 60
5.3 Designing Deterministic Finite State Machines 63
5.4 NondeterministicFSMs 66
5.5 From FSMs to Operational Systems 79
5.6 Simulators for FSMs* 80
5.7 Minimizing FSMs * 82
5.8 A Canonical Form for Regular Languages 94
5.9 Finite State Transducers * 96
5.10 Bidirectional Transducers * 98
5.11 Stochastic Finite Automata: Markov Models and HMMs * 101
5.12 Finite Automata, Infinite Strings: Büchi Automata * 115
Exercises 121
6Regular Expressions127
6.1 What is a Regular Expression? 128
6.2 Kleene's Theorem 133
6.3 Applications of Regular Expressions 147
6.4 Manipulating and Simplifying Regular Expressions 149
Exercises 151
7 Regular Grammars * 155
7.1 Definition of a Regular Grammar 155
7.2 Regular Grammars and Regular Languages 157
Exercises161
8Regular and Nonregular Languages162
8.1 How Many Regular Languages Are There? 162
8.2 Showing That a Language Is Regular 163
8.3 Some Important Closure Properties of Regular Languages 165
8.4 Showing That a Language is Not Regular 169
8.5 Exploiting Problem-Specific Knowledge 178
8.6 Functions on Regular Languages 179
Exercises 182
9Algorithms and Decision Procedures for Regular Languages187
9.1 Fundamental Decision Procedures 187
9.2 Summary of Algorithms and Decision Procedures 194
Exercises 196
10Summary and References198
PART III: CONTEXT-FREE LANGUAGES AND PUSHDOWN AUTOMATA 201
11Context-Free Grammars203
11.1 Introduction to Rewrite Systems and Grammars 203
11.2 Context-Free Grammars and Languages 207
11.3 Designing Context-Free Grammars 212
11.4 Simplifying Context-Free Grammars * 212
11.5 Proving That a Grammar is Correct * 215
11.6 Derivations and Parse Trees 218
11.7 Ambiguity 220
11.8 Normal Forms * 232
11.9 Island Grammars * 241
11.10 Stochastic Context-Free Grammars * 243
Exercises 245
12Pushdown Automata249
12.1 Definition of a (Nondeterministic) PDA 249
12.2 Deterministic and Nondeterministic PDAs 254
12.3 Equivalence of Context-Free Grammars and PDAs 260
12.4 Nondeterminism and Halting 274
12.5 Alternative Equivalent Definitions of a PDA * 275
12.6 Alternatives that are Not Equivalent to the PDA * 277
Exercises 277
13Context-Free and Noncontext-Free Languages279
13.1 Where Do the Context-Free Languages Fit in the Big Picture? 279
13.2 Showing That a Language is Context-Free 280
13.3 The Pumping Theorem for Context-Free Languages 281
13.4 Some Important Closure Properties of the Context-Free Languages 288
13.5 Deterministic Context-Free Languages * 295
13.6 Ogden's Lemma * 303
13.7 Parikh's Theorem * 306
13.8 Functions on Context-Free Languages * 308
Exercises 310
14Algorithms and Decision Procedures for Context-Free Languages314
14.1 The Decidable Questions 314
14.2 The Undecidable Questions 320
14.3 Summary of Algorithms and Decision Procedures 320
Exercises 322
15 Context-Free Parsing* 323
15.1 Lexical Analysis 325
15.2 Top-Down Parsing 327
15.3 Bottom-Up Parsing 340
15.4 Parsing Natural Languages 350
Exercises 358
16Summary and References360
PART IV: TURING MACHINES AND UNDECIDABILITY 363
17Turing Machines364
17.1 Definition, Notation and Examples 364
17.2 Computing With Turing Machines 375
17.3 Adding Multiple Tapes and Nondeterminism 382
17.4 Simulating a “Real” Computer * 393
17.5 Alternative Turing Machine Definitions 396
17.6 Encoding Turing Machines as Strings 400
17.7 The Universal Turing Machine 404
Exercises 407
18The Church-Turing Thesis411
18.1 The Thesis 411
18.2 Examples of Equivalent Formalisms * 414
Exercises 424
19The Unsolvability of the Halting Problem426
19.1 The Language H is Semidecidable but Not Decidable 428
19.2 Some Implications of the Undecidability of H 431
19.3 Back to Turing, Church, and the Entscheidungsproblem 432
Exercises 433
20Decidable and Semidecidable Languages435
20.1 D: The Big Picture 435
20.2 SD: The Big Picture 435
20.3 Subset Relationships between D and SD 437
20.4 The Classes D and SD Under Complement 438
20.5 Enumerating a Language 440
20.6 Summary 444
Exercises445
21Decidability and Undecidability Proofs448
21.1 Reduction 449
21.2 Using Reduction to Show that a Language is Not Decidable 452
21.3 Are All Questions About Turing Machines Undecidable? 466
21.4 Rice's Theorem * 468
21.5 Undecidable Questions About Real Programs 472
21.6 Showing That a Language is Not Semidecidable 474
21.7 Summary of D, SD/D and ØSD Languages 482
Exercises 483
22 Decidability of Languages That Do Not (Obviously) Ask Questions about Turing Machines* 487
22.1 Diophantine Equations and Hilbert's 10th Problem 488
22.2 Post Correspondence Problem 489
22.3 Tiling Problems 492
22.4 Logical Theories 495
22.5 Undecidable Problems about Context-Free Languages 499
Exercises 508
23 Unrestricted Grammars* 510
23.1 Definition and Examples 510
23.2 Equivalence of Unrestricted Grammars and Turing Machines 516
23.3 Grammars Compute Functions 518
23.4 Undecidable Problems About Unrestricted Grammars 521
23.5 The Word Problem for Semi-Thue Systems 522
Exercises 524
24 The Chomsky Hierarchy and Beyond* 526
24.1 The Context-Sensitive Languages 526
24.2 The Chomsky Hierarchy 539
24.3 Unification Grammars 540
24.4 Lindenmayer Systems 544
Exercises 553
25 Computable Functions* 555
25.1 What is a Computable Function? 555
25.2 Recursive Function Theory 565
25.3 The Recursion Theorem and its Use 573
Exercises 580
26Summary and References581
PART V: COMPLEXITY 585
27Introduction to the Analysis of Complexity586
27.1 The Traveling Salesman Problem 586
27.2 The Complexity Zoo 589
27.3 Characterizing Problems 590
27.4 Measuring Time and Space Complexity 593
27.5 Growth Rates of Functions 597
27.6 Asymptotic Dominance 598
27.7 Algorithmic Gaps 604
27.8 Examples * 605
Exercises 617
28Time Complexity Classes621
28.1 The Language Class P 621
28.2 The Language Class NP 633
28.3 Does P = NP? 642
28.4 Using Reduction in Complexity Proofs 644
28.5 NP-Completeness and the Cook-Levin Theorem 647
28.6 Other NP-Complete Problems 656
28.7 The Relationship between P and NP-Complete 672
28.8 The Language Class Co-NP 679
28.9 The Time Hierarchy Theorems, EXPTIME, and Beyond 681
28.10 The Problem Classes FP and FNP 689
Exercises 690
29Space Complexity Classes695
29.1 Analyzing Space Complexity 695
29.2 PSPACE, NPSPACE, and Savitch's Theorem 699
29.3 PSPACE Completeness 704
29.4 Sublinear Space Complexity 713
29.5 The Closure of Space Complexity Classes Under Complement 718
29.6 The Space Hierarchy Theorems 719
Exercises 720
30Practical Solutions for Hard Problems721
30.1 Approaches 721
30.2 Randomized Algorithms and the Problem Classes BPP, RP, ZPP 723
30.3 Heuristic Search 731
Exercises 740
31Summary and References742
APPENDICES745
AReview of Mathematical Background: Logic, Sets, Relations, Functions, and Proof Techniques745
A.1 Logic 745
A.2 Sets 753
A.3 Relations 756
A.4 Functions 769
A.5 Closures 776
A.6 Proof Techniques 779
A.7 Reasoning about Programs 792
A.8 A General Definition of Closure * 800
Exercises 804
BThe Theory: Working with Logical Formulas808
B.1 Working with Boolean Formulas 808
B.2 Working with First-Order Formulas: Clause Form and Resolution 820
B.3 Exercises 833
CThe Theory: Finite State Machines and Regular Languages835
DThe Theory: Context-Free Languages and PDAs839
D.1 Proof of the Greibach Normal Form Theorem 839
D.2 Proof that the DCFLs are Closed Under Complement 846
D.3 Proof of Parikh's Theorem 852
EThe Theory: Turing Machines and Undecidability856
E.1 Proof that Nondeterminism Does Not Add Power to Turing Machines 856
E.2 An Analysis of Iterative Deepening 861
E.3 The Power of Reduction 862
E.4 The Undecidability of the Post Correspondence Problem 864
FThe Theory: Complexity869
F.1 Asymptotic Dominance 869
F.2 The Linear Speedup Theorem 875
GApplications: Programming Languages and Compilers880
G.1 Defining the Syntax of Programming Languages 880
G.2 Are Programming Languages Context-Free? 883
G.3 Designing Programming Languages and Their Grammars 885
G.4 Compilers for Programming Languages 887
G.5 Functional Programming and the Lambda Calculus 890
HApplications: Tools for Programming, Databases and Software Engineering899
H.1 Proving Correctness Properties of Programs and Hardware 899
H.2 Statecharts: A Technique for Specifying Complex Systems 910
H.3 Model-Based Test Case Generation 913
H.4 Reverse Engineering 914
H.5 Normal Forms for Data and for Querying Relational Data Bases 916
IApplications: Networks918
I.1 Network Protocols918
I.2 Modeling Networks as Graphs 927
I.3 Exploiting Knowledge: The Semantic Web 929
JApplications: Security948
J.1 Physical Security Systems as FSMs 948
J.2 Computer System Safety 949
J.3 Cryptography 955
J.4 Hackers and Viruses 959
KApplications: Computational Biology962
K.1 A (Very) Short Introduction to Molecular Biology and Genetics 962
K.2 The Sequence Matching Problem 968
K.3 DNA and Protein Sequence Matching Using the Tools of Regular Languages 970
K.4 RNA Sequence Matching and Secondary Structure Prediction Using the Tools of Context-Free Languages 975
K.5 Complexity of the Algorithms Used in Computational Biology 977
LApplications: Natural Language Processing978
L.1 Morphological Analysis 978
L.2 Part of Speech Tagging 981
L.3 The Grammar of English 983
L.4 Building a Complete NL System 998
L.5 Speech Understanding Systems 999
MApplications: Artificial Intelligence and Computational Reasoning 1004
M.1 The Role of Search1006
M.2 A Logical Foundation for Artificial Intelligence 1007
M.3 A Rule-Based Foundation for Artificial Intelligence and Cognition 1022
NApplications: Entertainment: Music and Games1028
N.1 Music 1028
N.2 Classic Games and Puzzles 1034
N.3 Interactive Video Games 1046
OApplications: Using Regular Expressions1050
PUsing Finite State Machines and Transducers1054
P.1 Finite State Machines Predate Computers 1054
P.2 The Towers of Hanoi: Regular Doesn't Always Mean Tractable 1058
P.3 The Arithmetic Logic Unit (ALU) 1060
P.4 Controlling a Soccer-Playing Robot 1062
QApplications: Using Grammars1065
Q.1 Describing Artificial Languages Designed for Person/Machine Interaction 1066
Q.2 Describing Naturally Occurring Phenomena 1071
REFERENCES 1073
INDEX 1085


Close