Article Structure
Abstract
A system making optimal use of available information in incremental language comprehension might be expected to use linguistic knowledge together with current input to revise beliefs about previous input.
Introduction
In most formal theories of human sentence comprehension, input recognition and syntactic analysis are taken to be distinct processes, with the only feedback from syntax to recognition being prospective prediction of likely upcoming input (J urafsky, 1996; Narayanan and Jurafsky, 1998, 2002; Hale, 2001, 2006; Levy, 2008a).
Garden-path disambiguation under surprisal
The SURPRISAL THEORY of incremental sentence-processing difficulty (Hale, 2001; Levy, 2008a) posits that the cognitive effort required to process a given word w;- of a sentence in its context is given by the simple information-theoretic measure of the log of the inverse of the word’s conditional probability (also called its “surprisal” or “Shannon information content”) in its intra-sentential context w1,,,,,i_1 and extra-sentential context Ctxt:
Garden-pathing and input uncertainty
We now move on to cases where garden-pathing can apparently be blocked by only small changes to the surface input, which we will take as a starting point for developing an integrated theory of uncertain-input inference and surprisal.
Model instantiation and predictions
To determine the predictions of our uncertain-input/surprisal model for the above sentence types, we extracted a small grammar from the parsed
Empirical results
To test these predictions we conducted a word-by-word self-paced reading study, in which participants read by pressing a button to reveal each successive word in a sentence; times between button presses are recorded and analyzed as an index of incremental processing difficulty (Mitchell, 1984).
Conclusion
Language is redundant: the content of one part of a sentence carries predictive value both for what will precede and what will follow it.
Topics
conditional probability
- The SURPRISAL THEORY of incremental sentence-processing difficulty (Hale, 2001; Levy, 2008a) posits that the cognitive effort required to process a given word w;- of a sentence in its context is given by the simple information-theoretic measure of the log of the inverse of the word’s conditional probability (also called its “surprisal” or “Shannon information content”) in its intra-sentential context w1,,,,,i_1 and extra-sentential context Ctxt:Page 2, “Garden-path disambiguation under surprisal”
- Letting Tj range over the possible incremental syntactic analyses of words wlma preceding fell, under surprisal the conditional probability of the disambiguating continuation fell can be approximated asPage 2, “Garden-path disambiguation under surprisal”
- In a surprisal model with clean, veridical input, Fodor’s conclusion is exactly what is predicted: separating a verb from its direct object with a comma effectively never happens in edited, published written English, so the conditional probability of the NP analysis should be close to zero.2 When uncertainty about surface input is introduced, however—due to visual noise, imperfect memory representations, and/or beliefs about possible speaker error—analyses come into play in which some parts of the true string are treated as if they were absent.Page 3, “Garden-pathing and input uncertainty”
- Computing the surprisal incurred by the disambiguating element given an uncertain-input representation of the sentence involves a standard application of the definition of conditional probability (Hale, 2001):Page 6, “Model instantiation and predictions”
See all papers in Proc. ACL 2011 that mention conditional probability.
See all papers in Proc. ACL that mention conditional probability.
Back to top.
