Abstracts
Giovanni Buccino
http://www.unipr.it/arpa/mirror/english/staff/buccino.htm
How the motor system handles verbs and nouns
The embodiment approach to language is relatively recent and contrasts with a more classical view of language processing, where words are viewed as amodal. Embodiment theory claims that language understanding involves the activation of the same neural substrates, sensory and motor, activated when one experiences the action or the object to which a word refers. As for language referring to action, i.e. action verbs or sentences, a variety of behavioral and neurophysiological approaches have shown evidence for activation of the motor system during processing of action-related language material. For example there are behavioral studies showing slower reaction times when the effector used to respond is also involved in actual execution of the action expressed by the presented linguistic material. Other behavioural studies have demonstrated that the execution of a motor response is facilitated during the comprehension of sentences that describe actions taking place in the same direction as the motor response Together these findings were interpreted as resulting from an interaction between activation of the motor system for language understanding and activation for response.
Further evidence comes from neurophysiological and brain imaging techniques, EEG and fMRI, which showed that presentation of verbs associated with different effectors, results in somatotopic activation of motor areas and common activation in Broca’s area for action-related sentences and their action counterparts. There is also evidence from TMS work showing a modulation in the motor evoked potential for a muscle of a given effector, associated with the presented verb.
The mechanism through which words referring to actions, i.e. verbs, could elicit the motor representations for action itself can be explained in terms of the mirror neuron mechanism.
More recent evidence show that a similar modulation of the motor system also occurs during understanding of concrete nouns, as it is for concrete verbs. In a recent behavioral study nouns expressing objects related to hand, foot or abstract were used as stimuli, using a go-nogo paradigm with an early and delayed go-signal delivery. The results showed that participants at the early go-signal, gave slower right-hand responses for hand-related nouns compared to foot-related nouns. The opposite pattern was found for the left hand. These findings demonstrate an early lateralized modulation of the motor system during noun processing, most likely crucial for noun comprehension. More recently a Transcranial Magnetic Stimulation (TMS) study was carried out to compare modulation of the motor system when subjects read nouns referring to objects which are Artificial or Natural and which are Graspable or Ungraspable. TMS was applied to the primary motor cortex representation of the first dorsal interosseous (FDI) muscle of the right hand at 150ms after noun presentation. Analyses of Motor Evoked Potentials (MEPs) revealed that across the duration of the task, nouns referring to graspable artifacts (tools) were associated with significantly greater MEP areas. Analyses of the initial presentation of items revealed a main effect of graspability. Again the findings are in line with an embodied view of nouns, with MEP measures modulated according to whether nouns referred to natural objects or artifacts (tools), confirming tools as a special class of items in motor terms. Additionally these data support a difference for nouns expressing graspable versus non graspable objects, an effect which for natural objects is restricted to initial presentation of items. It may be hypothesized that as verbs recruit mirror neurons, nouns recruit canonical neurons, a set of neurons which, besides motor properties similar to those of mirror neurons, are sensitive to the presentation of objects.
Michiel van Elk
http://www.nici.ru.nl/anc/index.php?staff=vanelk
Mind the body in embodiment
The last decade cognitive neuroscience has seen a growing interest in an embodied view of cognition, according to which higher-level cognition is grounded in basic sensorimotor experiences. However, the embodied view of cognition has not gone unchallenged and has been criticized on both methodological and philosophical grounds. I will argue that many of the criticisms encountered by the embodied approach hinge on a cognitive interpretation of embodiment, that has often overlooked the fact that cognition takes place in a physical body acting in the real world. In other words: the embodied approach should put the body back in embodiment. To support this claim I will discuss developmental, behavioral and neuroimaging studies, relating different domains like perception, action, language and social cognition and highlighting the constitutive role of the body for cognition.
Raymond W. Gibbs, Jr.
http://people.ucsc.edu/~gibbs/
Embodied Experience and Metaphorical Meaning
Where do metaphors get their meanings? Contemporary theories of metaphor debate the motivations for metaphorical meanings in language. I will argue that a people’s recurring embodied experiences provide a significant and enduring constraint on the exact meanings many linguistic metaphors convey. I discuss the results of two studies, one a survey examining people’s mental imagery about their embodied experiences with paths and roads, with the second providing a corpus analysis of the ways path and road are metaphorically used in discourse. My hypothesis is that both people’s mental imagery for path and road, and speakers’ use of these words in metaphorical contexts are strongly guided by their embodied understandings of real-world events related to travel on paths and roads. The results of these studies demonstrate how bodily experiences with artifacts partly constrains not only how specific conceptual metaphors emerge, but how different metaphorical understandings are applied in talk about abstract entities and events. I use these findings to argue for a dynamical view of human performance in which embodied actions play a central role in creating metaphorical language.
Olaf Hauk
http://www.neuroscience.cam.ac.uk/directory/profile.php?olafhauk
Spatio-temporal patterns of brain activation reflect embodied action-word semantics
Neuroscientific theories of embodied semantics make strong predictions about brain activation patterns evoked by words. If understanding a word referring to an object or action is grounded in perceptual or motor experience, then the corresponding visual or motor areas of the brain should be activated in the process. More specifically, different cortical motor areas should be activated depending on the effector of the action (e.g. hand, leg or mouth). I will start with a selective review of functional magnetic resonance imaging (fMRI) studies that have confirmed these expectations. For example, stimuli referring to different bodily effectors (e.g. arm-, leg- or face-related words such as “pick”, “kick” or “lick”) produced somatotopic activation patterns in motor cortex. The findings have been extended to more specific categories of action-words, e.g. uni-manual action-words (“throw”) produced more strongly lateralized activation in motor cortex than bi-manual action-words (“clap”). fMRI methodology integrates brain activity over several seconds, and is therefore not able to distinguish between early retrieval and late post-translational processes. In order to address this problem, we showed that activation that is specific to action-words or object-words correlates negatively with word frequency, suggesting that this activation reflects retrieval of lexico-semantic information, rather than mental imagery. A more direct test, however, is to use electro- and magnetoencephalography (EEG, MEG) in order to measure brain activation with sufficient time resolution to distinguish early from late processes. In an ERP study, we found that visually presented arm-, leg- and face-related words could be distinguished as early as 210-230 ms after word onset, and that the putative generators of these signals showed a somatotopic pattern. This demonstrates that action-semantics affects early brain processes, but the evidence is still correlational: reading action words affects motor areas, but does activity in motor areas also affect action-word processing? In a transcranial magnetic stimulation (TMS) study, stimulation of hand and leg motor cortex at 150 ms facilitated arm- and leg-word processing, respectively, but only for the left hemisphere. In a combined EEG/MEG study, participants initiated trials by button press either with their finger or their foot. Words were presented while the button was still pressed, i.e. motor cortex was activated. We found effects of word-effector congruency (e.g. finger button press followed by arm-word is congruent, but followed by a leg-word incongruent) both in hand motor cortex and in posterior superior temporal gyrus, i.e. a classical perisylvian language area, around 150 ms. This demonstrates that motor cortex activation specifically affects action-word processing at early latencies. Finally, neuropsychological studies (e.g. in semantic dementia patients) have shown that brain impairment affects different action-word categories to different degrees.
We conclude that there is converging evidence from different metabolic and electrophysiological neuroimaging techniques as well as neuropsychology supporting the idea of rapid access to embodied semantic representations for action-words in language comprehension.
Srini Narayanan
http://www.icsi.berkeley.edu/~snarayan/
Simulation Semantics: A computational framework for exploring the links between communication, cognition and computation.
The Berkeley/ICSI NTL project is an ongoing attempt to model language behavior in a way that is both cognitively plausible and computationally practical. Work within these projects coupled with a variety of converging evidence from cognitive linguistics, psychology and neuroscience suggests that language understanding involves embodied enactment which we call "simulation semantics". Simulation semantics hypothesizes the mind as "simulating" the external world while functioning in it. The "simulation" takes the best-fitting model of the noisy linguistic input together with general knowledge and makes new inferences to figure out what the input means and to guide response. Monitoring the state of the external world, finding the best-fitting analysis, drawing inferences, and acting jointly constitute a dynamic ongoing interactive process. In terms of language processing, simulation semantics integrates probabilistic dynamic inference with deep semantic analysis based on construction grammar, frame semantics, and cognitive linguistics. This talk reports on a computational realization of the simulation semantics hypothesis and preliminary results on applying the model to vexing problems in language understanding. Based on these results and those from a variety of ongoing imaging and behavioral experiments, I will argue that simulation semantics provides a crucial bridge that ties the multi-disciplinary evidence together to produce new insights into the nature of language.
Gerard Steen
Sensory motor concepts in four registers of English - Exploring the VU Amsterdam Metaphor Corpus with Wmatrix
The idea that sensory-motor concepts are basic to the structure of our thought and language has given rise to new theoretical and empirical work in cognitive linguistics which ties in with the nature and function of metaphor as a cross-domain mapping facilitating the understanding of one thing in terms of something else (Lakoff & Johnson, 1980, 1999; Gibbs, 1994, 2006). The recent turn this cognitive-linguistic work has taken focuses on the relation between perception and meaning via the use of image schemas in primary metaphors (Mandler, 2004; Hampe, 2005), sensory–motor concepts playing a pivotal role in the theoretical and empirical arguments. One limitation with most of this work is that it has used a selection of relatively clear examples, including difficulties are burdens (heavy/ light taxes) or quantity is height (rising/soaring/plummeting unemployment). The present contribution aims to take a step forward by examining four relatively large datasets of metaphorical language use and raising the question how metaphorically used sensory-motor concepts behave ‘in the wild.’
The data come from the VU Amsterdam Metaphor Corpus, an annotated sample from the British National Corpus (Steen, Dorst, et al., 2010a). It comprises some 190,000 words divided across four registers of spoken and written English: conversation, fiction, news, and academic texts. Data analysis has shown that there is a substantial degree of interaction between register, word class and metaphor, with an average of 13.6% of all words being used metaphorically (Steen, Dorst, et al., 2010b).
In the present talk I will discuss the distribution of four semantic domains across these data: ‘Location and direction’, ‘Moving, coming, and going’, ‘Putting, pushing, pulling, transporting’, and ‘Sensory’. These are four categories distinguished by Wmatrix, a program assigning semantic tags to lexical units in natural text (Rayson, 2008). Application of Wmatrix to the VU Amsterdam Metaphor Corpus will enable close inspection of all metaphorical uses of all terms included in these three semantic domains. This will throw a usage-based light on the question how these large sets of linguistic forms can be seen to reflect a clear and fundamental role of sensory-motor concepts for modelling more abstract domains of experience.
Alex Tillas
Grounding Cognition: The Role of Language in Thinking
In this paper, I investigate the relationship between natural language and thinking. In doing so, I review and evaluate the main views in this debate, argue that none of the available views is satisfactory and suggest an alternative one. In particular, I adopt a view according to which thinking operates by and large according to associationistic rules and argue that natural language plays a crucial role in thinking but not a constitutive one, as it is often argued in the literature. The suggested view seems to enjoy significant independent empirical support, mainly from work done with aphasic subjects. The major challenges that all associationistic views of thinking face are the problems of propositional thinking and compositionality of thought. I briefly suggest how these challenges could be met in the light of the suggested view of thought production.
Keywords: Grounded Cognition; Language; Associationism; Aphasia; Concept Empiricism