QUAID: a questionnaire evaluation aid for survey methodologists.

QUAID (question-understanding aid) is a software tool that assists survey methodologists, social scientists, and designers of questionnaires in improving the wording, syntax, and semantics of questions. The tool identifies potential problems that respondents might have in comprehending the meaning of questions on questionnaires. These problems can be scrutinized by researchers when they revise questions to improve question comprehension and, thereby, enhance the reliability and validity of answers. QUAID was designed to identify nine classes of problems, but only five of these problems are addressed in this article: unfamiliar technical term, vague or imprecise relative term, vague or ambiguous noun phrase, complex syntax, and working memory overload. We compared the output of QUAID with ratings of language experts who evaluated a corpus of questions on the five classes of problems. The corpus consisted of 505 questions on 11 surveys developed by the U.S. Census Bureau. Analyses of hit rates, false alarm rates, d' scores, recall scores, and precision scores revealed that QUAID was able to identify these five problems with questions, although improvements in QUAID's performance are anticipated in future research and development.

Verification of statements about story worlds that deviate from normal conceptions of time: what is true about Einstein's Dreams?

College students read chapters from a novel written by Alan Lightman (Einstein's Dreams) and later provided verification judgments on the truth/falsity of test statements. Each chapter described a different fictional village that incorporated assumptions about time that deviate from our normal TIME schema, e.g., citizens knowing exactly when the world will end, time flowing backward instead of forward. These novel assumptions about time provided interesting insights about life and reality. In two experiments, we examined whether readers could accurately incorporate these novel assumptions about time in the fictional story worlds, as manifested in the verification judgments for statements after story comprehension. The test statements included verbatim typical, verbatim atypical, inference typical, and inference atypical information from the perspective of mundane reality that meshes with a normal TIME schema. Verification ratings were collected on a 6-point scale in Experiment 1, whereas Experiment 2 used a signal-response technique in which binary true/false decisions were extracted at-.5, 1.5, 3.5, 5.5, and 10.0 s. The college students were measured on literary expertise, reading skill, working memory span, and reading time. Readers with comparatively high literary expertise showed truth discrimination scores that were compatible with a schema copy plus tag model, which assumes that readers are good at detecting and remembering atypical verbatim information; this model predicts better (and faster) truth discrimination for verbatim atypical statements than for verbatim typical statements. In contrast, fast readers with comparatively low literary expertise were compatible with a filtering model; this model predicts that readers gloss over (or suppress) atypical verbatim information and show advantages for verbatim typical information. All groups of readers had trouble inferentially propagating the novel assumptions about time in a fictional story world, but the slower readers were more accurate in their verification of the atypical inferences. A construction-integration model could explain the interactions among literary expertise, reading time, and the typicality of test statements.

Discourse comprehension.

The field of discourse processing has dissected many of the levels of representation that are constructed when individuals read or listen to connected discourse. These levels include the surface code, the propositional textbase, the referential situation model, the communication context, and the discourse genre. Discourse psychologists have developed models that specify how these levels are mentally represented and how they are dynamically built during comprehension. This chapter focuses on the meaning representations that are constructed when adults read written text, such as literary stories, technical expository text, and experimenter-generated "textoids." Recent psychological models have attempted to account for the identification of referents of referring expressions (e.g. which person in the text does she refer to), the connection of explicit text segments, the establishment of local and global coherence, and the encoding of knowledge-based inferences.

Constructing inferences during narrative text comprehension.

The authors describe a constructionist theory that accounts for the knowledge-based inferences that are constructed when readers comprehend narrative text. Readers potentially generate a rich variety of inferences when they construct a referential situation model of what the text is about. The proposed constructionist theory specifies that some, but not all, of this information is constructed under most conditions of comprehension. The distinctive assumptions of the constructionist theory embrace a principle of search (or effort) after meaning. According to this principle, readers attempt to construct a meaning representation that addresses the reader's goals, that is coherent at both local and global levels, and that explains why actions, events, and states are mentioned in the text. This study reviews empirical evidence that addresses this theory and contrasts it with alternative theoretical frameworks.

Reading strategies of fast and slow readers.

In three subject-paced experiments we evaluated reading patterns at the word, line, and sentence level for fast and slow readers. A moving-window method was used to collect word reading times for natural texts. At the word level, reading times of word N were influenced by features of word N-1 for fast readers but not for slow readers. The lag effect exhibited by fast readers indicates that they continue to process a word when it is no longer in view, thus limiting the notion of immediate processing. Contrary to our initial expectation that fast readers would process only a single new argument from a sentence, whereas slow readers would process several new arguments, we found that both reader groups adopted a many-argument strategy. However, fast and slow readers differed in terms of the text units (lines vs. sentences) defining the new-argument effects: Fast readers exhibited greater new-argument effects relative to lines, whereas slow readers exhibited greater new-argument effects relative to sentences. Specifically, slow readers integrated the new arguments primarily at the end of the sentence, whereas fast readers did so at line boundaries. These results are discussed in terms of a buffer-and-integrate model of reading comprehension.

Processing of new arguments at clause boundaries.

In a subject-paced reading-time study, we examined the processing of new arguments at clause boundaries. Word reading times increased with the cumulative number of new-argument nouns at clause boundaries (as well as at sentence boundaries). New-argument nouns had a greater impact at clause boundaries than at nonboundary locations. In accordance with a buffer-integrate-purge model of reading (see Jarvella, 1979), the increase of reading times at boundaries was attributed to the integration of new information from the current sentence with prior information in the text representation. The increase at nonboundary locations was attributed to the growing load of buffering the new information. Reading times at clause boundaries were influenced to a greater extent by text-level integration than by such sentence-level processes as organization of words into clauses and linking of clauses within a sentence. The new contribution of this study was that it showed that clause boundaries provide an opportunity not only for sentence-level processes, but also for a text-level process, namely, the integration of text-new information with the growing text representation.

Tests of a holistic chunking model of sentence memory through analyses of noun intrusions.

A holistic chunking model of sentence acquisition and retrieval is described and tested by a prompted sentence recall procedure. In this procedure, subjects first study a list of unrelated sentences and later receive single-word prompts to cue sentence recall. The model assumes that (1) words in sentences are grouped into propositions during acquisition, (2) the propositions are encoded holistically and later retrieved as units, and (3) the retrieval of one proposition does not automatically lead to recovery of other propositions in a sentence. The model was tested by patterns of intrusion errors. Noun intrusions for elements within a recovered proposition were always related conceptually to the presented nouns, even when a noun violated the co-occurrence restrictions of the verb (e.g., the tray loved the house). In contrast, noun intrusions for elements outside of the scope of a recovered proposition were often unrelated to presented nouns. It was argued that patterns of intrusion errors provide more appropriate tests for sentence structure than do quantitative patterns of correct recall, at least from the framework of the holistic chunking model.