The dissertation has three main goals: (1) to show that a complete natural language processing (NLP) system can be built from distributed artificial neural networks, (2) to show that several high-level phenomena can be explained at the physical level using the special properties of these networks, and (3) to address various technical issues in connectionist cognitive modeling by developing new network architectures and techniques. DISCERN is a large-scale NLP system implemented entirely at the subsymbolic level. It learns to read short narratives about stereotypical event sequences, store them in episodic memory, generate fully expanded paraphrases of the narratives, and answer questions about them. The system input and output consists of sequences of orthographic word symbols, i.e. DISCERN processes sequential natural language. Several high-level phenomena emerge automatically from the special properties of distributed neural networks. DISCERN learns to infer unmentioned events and unspecified role fillers, and it generates expectations and defaults and exhibits plausible lexical access errors and memory interference behavior. Word semantics, memory organization and the appropriate script inferences are automatically extracted from examples. Processing in DISCERN is based on hierarchically-organized backpropagation modules, communicating through a central lexicon of word representations. The lexicon is a double feature map, which transforms the orthographic word symbol into its semantic representation and vice versa. The episodic memory is a hierarchy of feature maps, where memories are stored one-shot'' at different locations. Special mechanisms were developed for modular training, automatically developing distributed representations, role binding, type/token processing, lexical disambiguation, sequential communication, filtering out internal and external noise, many-to-many mapping, hierarchical self-organization and one-shot storage.
[ A much expanded version of this dissertation appeared as the book Subsymbolic Natural Language Processing: An Integrated Model of Scripts Lexicon, and Memory. ]
PhD Thesis, University of California. 334.