CS 343: Artificial Intelligence: Lecture Notes


Copyright © 2007 by Gordon S. Novak Jr.

Permission is granted for individuals to make copies of these notes for personal use, or for instructors to make copies for classroom use.

Note: Many of these pages use math symbols such as: &forall &exist . Microsoft Internet Explorer will not display the math symbols, but Netscape or Firefox will.

Index

1. CS 343 \ \ Artificial Intelligence

2. Artificial Intelligence as Science

3. Scientific Goals of AI

4. A.I. as Engineering

5. Perception

6. Robotics

7. Natural Language Understanding:

8. Planning

9. Expert Systems

10. Theorem Proving

11. Symbolic Mathematics

12. Game Playing

13. Characteristics of A.I. Programs

14. Symbolic Representation

15. Knowledge Representation

16. Semantic Networks / Frames

17. Logic

18. Logic ...

19. A.I. is the Future of Computing!

20. Advantages of Lisp

21. Lisp Code

22. Quick Lisp

23. Lisp Data

24. Quotation

25. Variable Values in Lisp

26. Constructing Lists

27. Extracting Parts of Lists

28. Combinations of CAR and CDR

29. Constructing List Structure

30. List Manipulation Functions

31. Substitution

32. Evaluation

33. Requesting Evaluation

34. Building Lists Incrementally

35. Building Lists Incrementally ...

36. Stepping Through A List

37. Predicates

38. More Predicates

39. Logical Operators

40. IF Statement

41. EQ Predicate

42. Membership Testing

43. Association Lists

44. Use of Recursion in Lisp

45. Examples of Recursion in Lisp

46. More Examples of Recursion

47. Binary Tree Recursion

48. SUBST is COPY-TREE with Substitution

49. Executing Multiple Statements

50. The LET Construct

51. Example of LET

52. Iteration Using DOLIST

53. Search

54. Why Search is Necessary

55. Outline of Search Topics

56. State Space Search

57. Tic-Tac-Toe as a State Space

58. Search Tree

59. Solving Problems Using Search

60. Basic Recursive Algorithm

65. Basic Depth-first Search Algorithm

66. Comments on Search Algorithm

67. Recursive Depth-First Search

68. State Space Search Program

69. Notes on State Space Search Program

70. Missionaries and Cannibals

71. Missionaries/Cannibals Search Graph

72. Missionaries and Cannibals Representation

73. Functions for Missionaries and Cannibals

74. Testing Missionaries and Cannibals

75. Combinatoric Explosion

76. Do Faster Computers Help?

77. Irrelevant Operators

78. Search Order

79. Search Order

80. Depth-First Search

81. Bounded Depth-First Search

82. Iterative Deepening

83. Cost of Iterative Deepening

84. Using Heuristics to Guide Search

85. Hill Climbing

86. Breadth-First Search

87. Breadth-First Search Algorithm

88. Uniform-Cost Search

89. Ordered Search

90. The Ordered Search Algorithm

91. Evaluation Functions

92. Heuristic Search: A*

93. Heuristic Search for Route Finding

94. Ordered Search for Route Finding

95. Effect of Heuristic Function

96. Admissibility of Heuristic Function

97. Informed Heuristic Functions

98. Features of Heuristic Functions

99. Heuristic Search Handles Local Maxima

100. Iterative Deepening A* (IDA*)

101. Beam Search: $MA^{*}$ and $SMA^{*}$

102. Forward vs. Backward Search

103. Search Tree vs. Search Graph

104. Problem Reduction Search

105. Problem Reduction Search: Flowchart

106. Problem Reduction Representations

107. AND/OR Graph Example

108. Solution to an AND/OR Graph

109. Search of an AND/OR Graph

110. Counterexamples

111. Searching Game Trees

112. Game Trees

113. Static Evaluation Functions

114. Minimax Evaluation

115. Alpha-Beta Search

116. Implementing Alpha-Beta Search

117. Alpha-Beta Search Example

118. Game Tree Search

119. Samuel's Checkers Program

120. General Problem Solver (GPS)

121. Problems with GPS

122. Searching in Abstraction Spaces

123. Genetic Algorithms

124. Outline of Genetic Algorithm

125. Constraint Satisfaction Problems

126. CSP as Backtracking Search

127. Constraint Network

128. Optimizing Search

129. Constraint Propagation

130. Other Techniques and Kinds of CSP

131. Difficulty of CSP

132. DENDRAL (Buchanan and Feigenbaum)

133. Underlying State Space:

134. Ways to Reduce Search Space: Heuristics

135. Goal Testing:

136. Search as a Basic Technique

137. Where Search Should Fit in an AI System

138. Knowledge Representation and Reasoning

139. Representation Hypothesis

140. Computation as Simulation

141. Alternatives to the Representation Hypothesis

142. Kinds of Knowledge

143. Knowledge Representation System

144. Knowledge Representation

145. Symbolic Representation

146. Retrieval: Matching Problem

147. Knowledge Representation Methods

148. Knowledge Representation: Hard Problems

149. Logic for Artificial Intelligence

150. Logical Representation

151. Propositional Logic

152. Interpretation in Propositional Logic

153. Equivalent Formula Laws

154. Ways to Prove Theorems

155. Backward Chaining

156. Backchaining Example

157. Forward Chaining

158. Forward Chaining Example

159. Resolution

160. Resolution for Propositional Calculus

161. Normal Forms

162. Logical Consequence

163. Resolution for Propositional Calculus

164. Resolution Step for Propositional Calculus

165. Resolution Step

166. Examples of Resolution Step

167. Example: Propositional Calculus Resolution

168. Satisfiability Checking

169. Predicate Calculus (First-order Logic)

170. Overview of Predicate Calculus Resolution

171. Prenex Normal Form

172. Order of Quantifiers

173. Skolemization

174. Standard Form

175. Proof by Contradiction

176. Herbrand's Theorem

177. Resolution for Predicate Calculus

178. Examples of Unification

179. Substitutions

180. Unification Algorithm

181. Unification Implementation

182. Simple Unification Algorithm

184. Soundness and Completeness

185. Resolution Strategies

186. Resolution Strategies ...

187. Resolution Example

188. Resolution Example

189. Resolution as Syntax

190. Natural Deduction

191. Backchaining Theorem Prover

193. Question Answering

194. Answer Extraction

195. Planning

196. The Frame Problem

197. Planning: Situation Calculus

198. Operators in Situation Calculus

199. Frame Axioms

200. Planning: STRIPS

201. STRIPS Operators for Blocks World

202. STRIPS: Operator Application

203. Selection of STRIPS Operators

204. Kinds of Planning

205. Criticality Number

206. Plan Monitoring

207. Weaknesses of A.I. Planning

208. Knowledge Rep. in Predicate Calculus

209. Rules

210. Forward Chaining

211. Backward Chaining

212. Importance of Backchaining

213. Reason Maintenance

214. Implementing Retraction

215. Truth Maintenance

216. ATMS

217. Closed World Assumption

218. PROLOG

219. Non-Monotonic Logic

220. Induction and Abduction

221. Predicate Calculus: Representation Language

223. Predicate Calculus as Programming Language

224. When to Use Logic

225. Semantic Networks and Frames

226. Property List Representation

227. Property Lists

228. Advantages of Property Lists

229. Frames

230. ``Frame'' Software Packages

231. Typical Features of Frames

232. Simple Frame Program

233. Example Frame Data

234. Shadowing

235. Multiple Inheritance

236. Example Frame

238. Advantages of Frames

239. Disadvantages of Frames

240. When to Use Frames

241. Object-Oriented Programming

242. Access to Objects

243. Internal Implementation is Hidden

244. Advantages of Objects

245. Object-Oriented Programming vs. Frames

246. Unique Features of Frames

247. Unique Features of Object-Oriented Systems

248. Example: Frame

249. Example: Object

250. Benefits of Frame Idea

251. Scripts

252. Ontology

253. Some Languages / Inference Engines

254. Knowledge Bases or Ontologies

255. Overview of Knowledge Representation

256. Situated Action

257. Natural Language Processing (NLP)

258. Why Study Natural Language?

259. Model of Natural Language Communication

260. Minimality of Natural Language

261. Zipf's Law

262. Areas of Natural Language

263. Computer Language Understanding

264. Problems in Understanding Language ...

265. Outline of Natural Language Section

266. Morphology

267. Lexicon

268. Lexical Features

269. Size of Lexicon

270. Statistical Natural Language Processing

271. Part-of-Speech Tagging

272. AI View of Syntax

273. Grammar

274. Language Generation

275. Parsing

276. Ambiguity

277. Sources of Ambiguity

278. Foreign Languages

279. Formal Syntax

280. Notation

281. Phrase Structure Grammar

282. Recognizing Automaton

283. Chomsky Hierarchy of Languages

284. Regular Languages

285. Context Free Languages

286. Context Sensitive Languages

287. What Kind of Language is English?

288. Parsing

289. Top-down Parser

290. Bottom-up Parsing

291. Chart Parser

292. Augmented Transition Networks

293. Augmented Transition Networks

294. Separability of Components

295. Problems with Separability

296. Combining Syntax and Semantics

297. How to Combine Syntax \& Semantics

298. Natural Language as an AI Problem

299. Semantics

300. Semantics ...

301. Ambiguity

302. Reducing Ambiguity

303. Case Theory

304. Case Relations

305. Case Relation Example

306. Case Frames

307. Disambiguation Using Case Frames

308. Preference Semantics

309. Pronoun Reference

310. Deep Semantics Influences Parsing

311. Reference

312. Referent Identification

313. Scripts and Reference

314. Internal Representation

315. How Not to do Representation

316. Tokens

317. Time Sequence

318. Rhetorical Relations

319. Conclusions on Semantics

320. Simple Language Processing: ELIZA

321. Spectrum of Language Descriptions

322. Semantic Grammar

323. Semantic Grammar: Extended Pattern Matching

324. Example Semantics for a Semantic Grammar

325. Compositional Semantics

326. ATN Written in Lisp

327. Sentence Pointer Handling

328. Word Category Testing

329. Lexicon Example

330. Database Access

331. Building Database Access

332. Parser

333. Phrase Parsing

334. Grammar Compiler

335. An Example Query

336. Example Queries

337. Natural Language Interfaces

338. Problems with NL Interfaces

339. Menu-based Natural Language

340. Statistical NLP

341. Speech Understanding

342. Machine Translation Example

343. Sentence Understanding in ISAAC

350. Preposition Semantics

351. Expert Systems

352. Power-Based Strategy

353. Expert Reasoning

354. Expert Knowledge

355. Choosing a Domain

356. Rule-Based Systems

357. Advantages of Rules

358. EMYCIN

359. Rule-Based Expert Systems

360. Production Systems (OPS-5 family)

361. Recognize-Act Cycle

362. Production System (OPS-5)

363. OPS-5 Data

364. OPS-5 Rules

365. Conflict Resolution

366. Evaluation of OPS-5

367. Reasoning Under Uncertainty

368. Bayes' Theorem

369. Uses of Bayes' Theorem

370. Joint Probability Distribution

371. Chain Rule

372. Bayesian Networks

373. Computing with Bayesian Networks

374. A Heretical View

375. EMYCIN's Certainty Factors

376. Certainty Factor Meaning

377. EMYCIN Data CF's

378. EMYCIN Antecedent CF

379. Rule Certainty Factors

380. Certainty Factor Combination

381. Certainty Factor Combination

382. Summary of CF Computations

383. Contradictions:

384. Duplicate Rules

385. Rule Subsumption

386. Increasing Certainty

387. EMYCIN CF vs. Probability Theory

388. Sensitivity Analysis

389. Expert Systems vs. Decision Trees

390. Machine Learning

391. Rule Induction

392. Sample Decision Tree:

393. Example of Rule Induction

394. Final Decision Tree with Classifications

395. Algorithm for Rule Induction

396. Alternatives for {\tt select-feature}

397. Limitations of Rule Induction

398. Digital Low-Pass Filter

399. Getting Knowledge From Expert

400. Interaction with Expert

401. Conceptual Islands

402. Advantages of Conceptual Islands

403. Expansion with Conceptual Islands

404. Orthogonal Knowledge Sources

405. Neuron

406. Neuron Firing

407. Function of Brain Regions

408. Somatotopy: Sensory and Motor

409. Early Experimental Work

410. Function from Brain Injury

411. Positron Emission Tomography

412. Micro-electrodes

413. Hubel and Wiesel: Visual Cortex

414. Visual Processing Areas

415. Vision

416. Mental Rotation

417. Somatotopic Processing

418. Machine Vision

419. Digitization

420. Low-Pass Filtering

421. High-Pass Filtering

422. Convolution

423. Feature Detection

424. Hough Transform

425. Generalized Hough Transform

426. Segmentation

427. Shape from Shading

428. Lambert's Cosine Law

429. Object Identification

430. Motion

431. Other Sensory Modes

432. Adding Semantics to Vision

433. Practical Vision

434. Theory and Practice

435. Driving a Car

436. Driving: Architecture

437. Driving: Detailed Architecture

438. Control of Spacecraft

439. Remote Agent: Architecture

440. Engine / Valve Configurations

441. Understanding Machines from Movies

442. Understanding Machines: Example


CS 343