A Connectionist Model of Sentence Comprehension and Production

Douglas Rohde, Ph.D.
Post-Doctoral Researcher
Massachusetts Institute of Technology

January 17, 2003
11 a.m. - 12 noon
Fuller Labs 320

Abstract

Inspired by the work of Chomsky, and in a tradition exemplified more recently in the work of Pinker, linguists have largely viewed natural language processing as based on a system of abstract variables and rules for manipulating them. Such a "symbolic" approach is well suited for describing syntactic grammars, but it tends to characterize language properties as all-or-none and has minimized the roles of statistical frequency and semantics, or meaning. As a consequence of this, the symbolic approach faces a learnability problem. In the absence of frequency, one can prove (according to certain assumptions) that language could not be learned on the basis of the input that children receive. Therefore, many researchers conclude that most of the linguistic principles that children might require to acquire a language must be built into the genes in the form of a Universal Grammar.

Recent work in psycholinguistics demonstrates that frequency and semantic influences play important and immediate roles in sentence processing. This is best demonstrated in the context of temporary ambiguities. Sentence 1 will most likely seem ungrammatical on first reading because it is not obvious that "raced past the barn" is a reduced relative clause and not the main verb phrase of the sentence.

The horse raced past the barn fell
The evidence examined by the lawyer had been fabricated.

Sentence 2 has a similar structure, but is much easier to parse. The reason is that we are sensitive to subtle factors, such as the fact that "evidence" is rarely the agent of an action, and "examined" is more likely to be used passively than "raced". These sorts of lexical frequencies are not easily incorporated into a symbolic model and such models are generally unable to learn languages from the types of input a child might receive without having a large amount of linguistic knowledge built in.

I will discuss an alternative view of language processing exemplified in a connectionist (or neural network) model of sentence comprehension and production. The model has been trained to comprehend and produce sentences from a fairly complex subset of English. The model is able to learn from realistic inputs without any specific linguistic knowledge built-in in advance. It also demonstrates sensitivity to various statistical and semantic properties of the language to help it resolve temporary ambiguities as in sentence 2. The model is evaluated by comparing its performance to that of humans on various reading and production tasks. I will describe the model, present some of these analyses, and talk about the role this form of computer modeling plays in bridging the gap between our understanding of neurons and high-level cognitive behavior.

Host

Prof. Neil Heffernan

Refreshments will be served in FL 320 beginning at 10:50 a.m.

Maintained by webmaster@wpi.edu
Last modified: Sep 27, 2006, 16:05 EDT