The origin of language in gesture–speech unity

Part 5: The dynamic dimension, modes of consciousness.

David McNeill, University of Chicago

The dual semiosis of global-synthetic gesture, merging with analytic-combinatoric speech, synchronizing at points where they are co-expressive – namely, gesture–speech unity – led to other dynamic properties: the imagery–language dialectic, and three others, “psychological predicates,” “communicative dynamism,” and that GPs self-unpack by “calling” constructions to do it.

Collectively these properties comprise “the sentence” viewed dynamically.  Dynamic properties arose organically out of Mead’s Loop. They would not have been separately selected. They are among the “new actions” mentioned in Part 4, are themselves linked and are inseparable from context. The context, dynamic in itself, penetrates GPs and leads ineluctably to dynamic properties.

We will also see that Wundt’s two consciousnesses of the sentence, the “simultaneous” and the “sequential,” plus a third to be introduced, “metapragmatic consciousness,” relate to the dynamic dimension while focusing on different aspects of it.

1. The dialectic. Mead’s Loop was the author of the dialectic, by bringing the gesture, where the semiotic is global and synthetic, into unity with speech, where cultural ratification and shareability create the opposite semiotic of analysis and syntagmatic value. It also explains why gesture-speech synchrony occurs: synchrony is a matter of thought in the dialectic; it is not an accident or the result of signals back and forth, but an intrinsic part of thinking and language.

2. The psychological predicate (as opposed to a grammatical predicate) is the newsworthy content in a field of contextual possibilities or “equivalents” (Jakobson).  The gesture–speech unit is equally a discourse unit.  It has absorbed its context as a matter of its formation. A psychological predicate:

  • Marks a significant departure in the immediate context; and
  • Implies this context as background.

One of Vygotsky’s examples is a crashing clock (p. 250): There is a crash in the next room – someone asks: “what fell?” (the answer: “the clock”), or: “what happened to the clock?” (“it fell”). Depending on the context – here crystallized in the questions – the newsworthy reply (the psychological predicate) highlights different elements.

This logic also applies to the GP. In forming a GP, the speaker shapes the background in a certain way, in order to highlight an intended differentiation within it, much as the questioner about the falling clock shaped the context of the replies.  We trace this logic using the following examples:

The (a) illustration (also shown in Part 4) is a GP in mid-flight (just as the speaker was saying the extra-stressed vowel of “through”), “he goes up thróugh the pipe this time,” with the gesture during the boldfaced portion.

The speaker had just before described how Sylvester had climbed the pipe on the outside (with the usual catastrophic outcome), so that he was going up it now on the inside was newsworthy. Her psychological predicate was this information.

The context of equivalents it differentiated after the previous mention of the pipe was approximately WAYS OF CLIMBING A PIPE TO REACH TWEETY. Regarded dynamically, stemming from Mead’s Loop, this context and psychological predicate, ON THE INSIDE, were as much component parts of “he goes up thróugh the pipe this time” as were the words themselves.

As before, gesture and speech are co-expressive and semiotically opposites. The gesture depicted the rising character, his direction of movement, and that his path was inside the pipe; a collection of meanings we labeled “rising hollowness,” and depicted them in a single symbol.  Speech at the same time depicted the content analytically – broken into segments (he, goes, up, and through) and relating the segments by combination. The stress on “through” with extra effort was co-expressive with the added effort of the gesture’s spread open hand. In this way, the GP, as a unit, was comprised of synchronous and co-expressive semiotic opposites.

It was this content, differentiated in the psychological predicate and alive throughout the utterance, that filled in part Wundt’s “simultaneous” consciousness.

3. Communicative dynamism (a concept from Firbas) is the force pushing communication forward. As a dynamic feature it too emerged from Mead’s Loop and is also part of simultaneous consciousness.

In fact, the psychological predicate and communicative dynamism are deeply connected, actu­ally are the same thing, a GP looking backwards or forwards at its context.  It is a psycho­logical predicate as the GP looks at the context that it differentiates, a context possibly shaped to make the differentiation possible; it is communicative dyna­mism as the GP looks forward at the new context that by its own differentiation it reshapes.

This is possible because a GP absorbs the context as a matter of its formation. The context is continu­ously running through it, being both differentiated and reconfigured as the GP absorbs it.

For the psychological predicate the maxim issignificance” = “two things.”  “One” meaning is (a) a point of differentiation and (b) the context in which it is differentiated; a GP is both.  Merely having an association or intending a meaning, etc. is not enough.

For communicative dynamism the maxim is “more significant” = “more effort,” effort being the material carrier of the force pushing the communication forward (see Part 4 for the material carrier). It is not that the gesture expands as speech shrinks (as one might think if gesture took over from speech, but gesture and speech are unified). The most elaborate linguistic units are accompanied by the most developed gestures, the least with the least. The more discontinuous an utterance from the previous context, the more probable a gesture, the more internally complex it will be, the more complex the synchronous speech, and the more all of it adds to communicative dynamism.

Communicative dynamism is the opposite of an economic, costs-minimizing/benefits-maximizing, “least effort” = “best result” dictum that some claim to be the optimal form of action. The economic model, for those fond of eponymous acronyms, can be called “Summer Lazing on the Beach” or SLOB: so minimal effort, “Ugh!” brings maximal benefit, Jeeves: “Would you care for a cool drink, sir?”  However, the communicative dynamism of the GP is the opposite of SLOB.

A natural experiment featuring psychological predicates and communicative dyna­mism shows these two faces of the GP at work.

Fortui­tously, in the cartoon stimulus, Sylvester uses the drainpipe twice to reach Tweety. So far we’ve concentrated on the second of his attempts, the inside ascent. Just before it, in the cartoon, he climbed the pipe on the outside, like a ladder.  Comparing descriptions of the two ascents shows how the context is divided between psychological predicate and communicative dynamism.  Whatever a psychological predicate differentiates, communicative dynamism pushes forward, and these are the GP’s two faces (the natural experiment was discovered by my colleague, Susan Duncan).

If a speaker recalls both attempts, in the correct outside-inside order, as did (a), the psychological predicate relating to the second attempt should focus on interiority, reflecting the effects on context of the previous GP’s communicative dynamism. This follows from the psychological predicate concept. With the second attempt, climbing itself is no longer newsworthy; interiority is now the newsworthy point, and we get ON THE INSIDE, the psychological predicate orchestrating Speaker (a)’s “rising hollowness” growth point.

However, if a speaker recalls only the inside attempt and fails to recall the outside attempt, or recalls both attempts but reverses their order, the context would not have been reshaped by the outside ascent.  Then the interiority of the inside ascent would not be newsworthy, it lacks equivalents to which it contrasts and would not be part of a new psychological predicate.

Speaker (b) illustrates this situation. She had forgotten the outside ascent and, for her, the context of the inside ascent was not WAYS OF CLIMBING THE PIPE,with a psychological predicate of ON THE INSIDE, but WAYS OF USING A DRAINPIPE,with CLIMB IT the newsworthy point.  That is, not having the communicative dynamism of the outside’s CLIMB IT, this very context and psychological predicate now arise for the inside climb.

This also follows from the psychological predicate concept. Lacking discourse significance, interiority would be just another detail from the cartoon (no one in any experiment has ever recalled only the outside attempt).

And as the (b) illustration shows, the gesture for the inside ascent depicted a simple ascent, with no interiority. Speech likewise, co-expressively, contained only CLIMB IT (“he tries climbing …”).

We can be certain Speaker (b) knew that Sylvester had climbed the pipe on the inside since she continued her description with an allusion to it, saying “Tweety … drops a bowling ball down the rain barrel”.

This natural experiment shows that Speakers (a) and (b) had different communicative dynamisms for the same cartoon event, correlated with their two psychological predicates, and although the event was the same were experiencing different simultaneous consciousnesses while describing it.

4. GP’s self-unpacking.  The essence of unpacking is: (1) without contradiction, to render a GP into a communicable, socioculturally mandated construction; and (2) to use the sense of well-formedness this construction brings as the GP’s stop-order.  A construction is the stable form on which a GP comes to rest.

Wundt’s “sequential consciousness” is awareness of this unpacking process while it is going on.

The GP core and its unpacking are on different functional levels, yet are intimately tied. This is because unpacking is a kind of self-unpacking by a GP.  A GP does it by calling forth a construction and this can take place at any point, while or after the GP forms.

Something similar could have existed at the dawn. Primordial GPs, seeking dialectic resolutions, produced pressures to develop ways of unpacking them. GPs and self-unpacking could have evolved together.

Both successes and failures of unpacking can be traced to a common source, the need to find constructions with semantic values that mesh with (or at least do not violate) the intended point of differentiation of a GP.

An example of an unpacking going awry is what Nancy Dray and I called the “nurturing example”: a would-be unpacking that contradicted the intended meaning. A speaker in a conversation with a friend was attempting to convey a nuanced idea, that a third person she was describing was given to performing nurturing acts, but these good deeds were also intrusive, cloying, and unwelcome. Initial false starts were based on trying to unpack this idea with “nurture” as a transitive verb (she would “nurture” someone). The effort was repeatedly rejected.

Ultimately an oblique construction that circumvented transitivity was found in (2):

(1) The fact [that she’s . . . ] [she’s nu- uh]

(2) [ . . . she’s somehow. . . she’s] [done this nurtur] [ing] [thing and here you]

The problem was a mismatch at (1) of the transitive construction to the context being differentiated. Transitivity means, roughly, that the woman described has a direct transformative impact via nurturing on the recipient of her action. However, this meaning distorted the idea the speaker intended to convey, which was something more like HER OTIOSE ACTS. The slight but successful updating in (2) separated effect from act and gave an unpacking that differentiated this context without clashing with it.

An example of a success, by yet another speaker retelling the Sylvester and Tweety cartoon, is illustrated in the following:

The drawings show two phases of a downward thrusting gesture for Tweety’s launch of the bowling ball: first (left), the beginning of the gesture (the start of its preparation), which we conclude was the GP’s onflash (there is no other reason the hands took this shape and position than to get ready to perform the gesture). Then (middle), the gesture stroke surfacing with its co-expressive speech:

  1. And Tweety Bird runs and gets a bowling ba[ll (bracket where preparation starts)
  2. And drops it down the drainpipe” (boldface = the gesture stroke).

The “it down” plus gesture GP core unpacked itself by calling a construction in which the Tweety Good-force could be transferred to the bowling ball. This was “caused-motion,” where Tweety appears as agent and causes a change of state of the bowling ball. The change was two-fold, both causing the bowling ball to move and, metaphorically, turning it into Tweety’s surrogate.

A construction has its own meaning, pre-analyzed and broken into slots where words or other constructions are inserted with pre-determined syntagmatic values.

With caused-motion there is a slot for the agent of change (Tweety) and for the object undergoing the change (the bowling ball).  Thus the GP had points of contact in the construction by which it could “call” it.

To do this however the speaker had to transform the cartoon to make Tweety, not gravity, the agent.  We see this transformation at the very start. The gesture was shaped, at the onset of preparation, to “take up a position in the world of meanings” (Merleau-Ponty’s remark, quoted in Part 3), a meaning-world having to do with caused-motion, agency and unpacking.

The handshape was not an iconic depiction of the cartoon event – there, as seen in the last panel, Tweety supported the bowling ball with his “hands” facing upward and released it, letting gravity do the rest.  In the gesture the handshape was rather that of an agent thrusting a curved surface down. It was with this agent- and object-readiness the GP reached out to caused-motion.

5. Metapragmatic awareness. The simultaneous and sequential are consciousness of the sentence on the inside. Metapragmatic awareness is consciousness of the external forces shaping, guiding and motivating it, including context.

The term metapragmatics is from Michael Silverstein.  Its essential feature is awareness of the pragmatic effects of one’s speech in the social/communicative situation.

Metapragmatic consciousness stems from the social reference that Mead’s Loop provides. Without a sense of one’s own actions as social and public, metapragmatic indicators would not arise.

To bring Silverstein’s emphasis on structural indicators into our discussion of the dynamic dimension, I slightly modify his terminology to “metapragmatic orchestration.”

Caused-motion. To understand caused-motion metaphorically, not only to effect a change of the bowling ball’s location but to change its character from object to a moral force, metapragmatic orchestration of the construction by the speaker’s paradigm of opposed forces took place, Tweety–the Good–down versus Sylvester–the Bad–up.

The plurifunctional “it” had the expected cohesive effect of tying together the two references to the bowling ball in (1) and (2) but it also reflected a metapragmatic presupposition that objects introduced at one point, bowling balls, do not vanish a moment later. This presupposition orchestrated the pronoun’s double role in the unpacking.

The “it” reflected, not just anaphora, which it does (and is important for how the “it” could carry the assumption that the bowling ball did not vanish but continued in new form), but also that the bowling ball was changing from an object into the force for Good against Sylvester. All of this reflects the speaker’s construal of the episode as a paradigm of antagonistic forces and, more broadly, as Silverstein, after Halliday and Hasan, terms it, its “delicacy.”

What to say. Metapragmatics told the speaker that the bowling ball had to be mentioned, that the GP of one event (the outside ascent) had created a context the next GP (the inside ascent) was able to differentiate, etc.

It also accounts for “drops,” which, while part of caused-motion, allowed the motion of the bowling ball to stem from gravity, over “throws” or “thrusts,” which did not.

And, finally, this current-day sentence, with all its richness beyond denotative content, models the many selection pressures (obviously not the content) that GPs could have radiated at the dawn.

Summary: How Mead’s Loop engendered GP properties.

I conclude by listing the chief properties of the dynamic dimension and GP, and how Mead’s Loop could have engendered them.

1. The dual semiotic dialectic.  The new form of mirror neuron response in Mead’s Loop merged vocal movements and gesture, and synchronized them at points where they were co-expressive of the same underlying idea, laying the ground for the imagery–language dialectic and the dynamic dimension of language. Once codified linguistic forms evolved (and this would have been immediate, along with GPs) a dialectic would be the response. Without Mead’s Loop, gesture and speech would have had only the loose connection seen with pantomime, and language could not have escaped the single-semiosis box.

2. The social reference.  The social orientation of mirror neurons with the Mead’s Loop “twist” gave gestures and GPs a social reference character. Without Mead’s Loop gestures could have social reference only if directed at an interlocutor. But speech-unified gesticulations are not necessarily aimed at someone (and except for emblems and points, both culturally mandated to include interpersonal phatic connections, they rarely are) yet gestures and their GPs are social (“public”) entities. Also, crucially, a foundation in Mead’s Loop opened a route over which the social conventions of speech and thought could form, GPs absorbing social-interactive content. An important effect of the inherent sociality of GPs due to Mead’s Loop arose in the origin of syntax, namely, “sharaeability.”

3. Origin of psychological predicates. The psychological predicate, the differentiation of what the speaker deems newsworthy in the immediate context, was inherent to Mead’s Loop by virtue of how it brought gesture in as a speech-orchestrating force. The inseparability of the psychological predicate from context resides in Mead’s Loop’s self-response. Similarly for the other direction of the GP’s regard, communicative dynamism.

4. Origin of catchments. Catchments are threads of consistent imagery attached to a discourse theme. They also arose with Mead’s Loop as a matter of course. Mead’s Loop binds imagery with speech and brings the meaning of the image into the (“twisted”) mirror neuron circuit. Although not shown, each time the “it down” speaker regarded the bowling ball as an antagonistic force its ball-shaped imagery returned along with the theme (a half dozen such occurrences).

5. Metapragmatic indicators.  Mead’s Loop engendered gesture–speech unities and also the metapragmatic scaffolding that encompasses the GP dialectic.  The social reference of gesture–speech unity makes the pragmatic effects of one’s own symbolic actions stand out and sets up conditions for metapragmatic indicators.

6. The equiprimordiality of speech and gesture. To select Mead’s Loop, speech and gesture had to evolve together. One could not have come first and the other later. This basic difference from gesture-first is perhaps the one step, if we try to single out one, that sets human linguistic evolution apart. Some avian species (crows, ravens) have evolved surprisingly elaborate vocal and gestural repertoires but have not taken the step that led to language, evolving a unit that is both sound production and gesture integrally. The entire Mead’s Loop process involves one’s own gesture impinging on the same area of the brain where vocal actions are orchestrated, so both are necessary.

7. Origin of unpacking. The GP is the core idea at the moment of speaking, differentiating a context; unpacking cradles it and intuitions of well-formedness comprise the stop-order. Even if simultaneous GP and unpacking are functionally distinct, with unpacking dependent on the GP.  This was they key to the origin of unpacking as well.  By combining imagery with codified form, a self-created selection pressure for the static dimension also arose.

But not theory of mind. Mead’s Loop however is not theory of mind. They are in fact opposites. The Mead’s Loop adaptation brings self-awareness of one’s own behavior as social, not a theory of the cognitions and intentions of another (also, a theory of mind would tap straight mirror neurons; it could work with Mead’s Loop but could not duplicate it).

Both theory of mind and Mead’s Loop depend on a more fundamental faculty, self-aware agency, a topic the late Susan Hurley explored in depth. The origin of Mead’s Loop depended on this sense. It is through awareness of one’s own agency that gestures can be responded to as if from another. Awareness of self as agent develops in the child around age 4, and both Mead’s Loop and theory of mind show themselves then as well (children’s language was discussed in Part 4).

Further reading

Dray, N. L. and McNeill, D. 1990.  ‘Gestures during discourse: The contextual structuring of thought,’ in S. L. Tsohatzidis (ed.), Meanings and prototypes: Studies in linguistic categorization, pp. 465-487. Routledge.

Firbas, Jan. 1971.  ‘On the concept of communicative dynamism in the theory of functional sentence perspective.’  Philologica Pragensia 8: 135-144.

Givón, Talmy. 1985. ‘Iconicity, isomorphism and non-arbitrary coding in syntax’, in J. Haiman (ed.). Iconicity in Syntax, pp. 187-219. Benjamins.

Goldberg, Adele. 1995.  Constructions: A construction approach to argument structure. Chicago.

Halliday, M.A.K. and Hasan, Ruqaiya. 1976. Cohesion in English. Longman.

Hurley, Susan. 1998. Consciousness in Action. Harvard.

Jakobson, Roman. 1960.  Concluding statement: Linguistics and poetics, In Sebeok, T. (ed.). Style in Language, pp. 350-377. MIT.

Pika, Simone and Bugnyar, Thomas. 2011. ‘The use of referential gestures in ravens (Corvus corax) in the wild.’ Nature Communications 29 November.

Silverstein, Michael. 2003. ‘Indexical order and the dialectics of sociolinguistic life.’   Language & Communication 23:193–229.

van Eijck, Jan and Visser, Albert, ‘Dynamic semantics,’ The Stanford Encyclopedia of Philosophy (Fall 2010 Edition), Edward N. Zalta (ed.), URL = <> (accessed 19 Nov. 2012).

Vygotsky, Lev S. 1987. Thought and Language. Edited and translated by E. Hanfmann and G. Vakar (revised and edited by A. Kozulin). MIT.


His new title How Language Began: Gesture and Speech in Human Evolution is now available from Cambridge University Press at £19.99/$36.99

1 comment to The origin of language in gesture–speech unity

  • We’re a group of volunteers and opening a new scheme
    in our community. Your website offered us with valuable info to work on. You have
    done an impressive job and our whole community will be grateful to you.

Leave a Reply




You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>