Overview

My research investigates the mechanisms and principles of language processing (particularly spoken language) using converging evidence from behavioral, eye-tracking, and computational modeling techniques. I am currently pursuing three inter-related research projects aimed at answering questions at different levels of language processing: (1) How is semantic information represented and accessed during word processing? (2) What is the cognitive basis of individual differences in spoken language processing? (3) How does context influence perception and comprehension of speech? For each project, computational modeling plays a key role as a crucial tool for theory development and testing.

Full list of publications and recent presentations

Collaborators

Semantic representation and access

Since comprehension is ultimate goal of language perception, understanding how word meanings are represented and accessed is critical to understanding language processing. Past studies of semantic structure have relied on investigations of pair-wise similarity using techniques such as priming. I examined the effects of neighborhood size as an alternative, larger scale approach to study the structure of semantic representations (Mirman & Magnuson, 2008). I found simultaneous inhibitory effects of dense near neighborhoods (many highly similar concepts) and facilitative effects of dense distant neighborhoods (many moderately similar concepts) for visually-presented words. This pattern conflicts with existing models of neighborhood effects, which assume that neighborhoods must have singular effects - either inhibitory or facilitative - or assign conflicting effects to different levels of processing.

In models based on attractor dynamics, each concept is represented by stable state on an energy surface and complex interactions can arise from the specific topographical relationship of attractors to each other. With respect to the behavioral data, I found that attractor models correctly predict that near neighbors should have inhibitory effects and distant neighbors should have facilitative effects on semantic access (Mirman & Magnuson, 2008). Thus, attractor dynamics account for contrasting effects of near and distant neighborhoods that could not be explained by traditional views of semantic structure.

Current eye-tracking and computational studies examine the time course of activation of near and distant semantic neighbors. Other studies examine the similarity structure of semantic representations by comparing concepts that share different types of features, with particular emphasis on distinctive vs. shared features (that is, features that are shared among many concepts, like "breathes" and features that are highly distinctive, like "moos"). The role of distinctive features in computing word meaning is an important topic of debate among theories of semantics (e.g., Randall et al., 2004; Cree et al., 2006) and my experiments will provide converging evidence regarding the role of feature distinctiveness in structuring semantic space.

Individual differences

Because language processing is so rapid and so complex, investigating language processing requires fine-grained experimental methods and complimentary cognitive modeling. Eye-tracking in the visual world paradigm provides fine-grained time course data about language processing, but traditional statistical methods have failed to take advantage of these data. I have worked to develop a statistical method based on polynomial regression and hierarchical linear modeling that effectively quantifies visual world paradigm data for testing experimental manipulation effects as well as individual differences (Mirman, Dixon, & Magnuson, 2008). I use the TRACE model of speech perception (McClelland & Elman, 1986) to interpret the experimental results in cognitive terms and test hypotheses about the underlying mechanisms.

The combination of statistical and computational modeling is particularly important in the case of individual differences. The statistical model effectively describes the data pattern and the computational model sheds light on the nature of language processing and the cognitive interpretation, which may help to develop remediation strategies. An initial analysis of the time course of spoken word activation and competition in typical college-age adults suggested that individual differences were due to differences in vocabulary size. Current studies aim to test this hypothesis directly. Other current projects examine the time course of spoken word activation and competition in patients with aphasia - an acquired impairment of spoken language processing. My analyses suggest that both Broca's and Wernicke's aphasics are slower to activate word representations, but for different reasons. I am using simulations of the TRACE model to test hypotheses about the computational basis of these impairments.

I am also beginning to extend this approach, and combine it with other behavioral methods, to investigate the cognitive basis of specific language impairment in children and adults. In preliminary research related to this project we have found that word segmentation based on transitional probabilities is linked to word learning in adults (Mirman, Magnuson, Graf Estes, & Dixon, 2008) and we will be examining individual differences in this link.

Context effects

It is amazing that we are able to understand speech considering the tremendous complexity, variability, and noisiness of the speech signal. The perceptual challenge can be somewhat alleviated by using context to constrain the possible percepts. I have investigated lexical, semantic, attentional, and global non-linguistic context effects and the common theme across these studies is that context integration occurs by interactive, graded constraint satisfaction. That is, all levels of processing contribute to perception of a stimulus (processing is interactive) and no single level dominates processing (interaction is graded).

One key debate has focused on whether word-level knowledge influences perception of speech sounds. Along with the work of others, my research demonstrates that word-level knowledge and processing interactively influence speech recognition (reviewed in McClelland, Mirman, & Holt, 2006) and provides insights into resolving debates about context effects in other domains, such as the influence of sentence context in determining the meaning of an ambiguous word like "bark" (Mirman, in press). Interactivity allows the system to tune itself in response to systematic changes in the input (Mirman, McClelland, & Holt, 2006) and improves recognition, especially under noisy conditions (Magnuson, Strauss, Harris, & Mirman, under review), though it can have detrimental consequences when bottom-up and top-down information conflict (Mirman, McClelland, & Holt, 2005).

Attention affects all levels of perceptual and cognitive processing, yet the role of attention in speech perception has been largely ignored. In a general interactive graded constraint satisfaction framework, attentional state can be considered as one type of context that influences processing. In a behavioral study, I found that attention to word-level information modulates the strength of word-level effects on speech perception. Inspired by neurophysiological studies of visual attention, I developed a computational framework for attention effects on language processing based on selective damping of different representations and showed that this framework correctly accounts for attentional modulation of lexical effects on phoneme recognition and other aspects of language processing (Mirman, McClelland, Holt, & Magnuson, 2008). Extending this notion of attention to the task of resolving ambiguity due to multiple meanings for a single word, I found that listeners can use global non-linguistic contexts to shift attention to particular regions of semantic space (Mirman, Magnuson, Strauss, & Dixon, 2008). In this study, I found that the pragmatic expectation that only concrete nouns will be presented causes participants to damp the activation of meanings that were not concrete nouns.

Collaborators: Past and present

Jim Magnuson (University of Connecticut & Haskins Laboratories)
J. Dixon (University of Connecticut)
Eiling Yee (University of Pennsylvania)
Sheila Blumstein (Brown University)
Katie Graf Estes (University of California, Davis)
Nicole Landi (University of Minnesota & Haskins Laboratories)
Lori Holt (Carnegie Mellon University)
Jay McClelland (Stanford University)
Michael Spivey (Cornell University)