Common coding theory

Common coding theory is a cognitive psychology theory describing how perceptual representations (e.g. of things we can see and hear) and motor representations (e.g. of hand actions) are linked. The theory claims that there is a shared representation (a common code) for both perception and action. More important, seeing an event activates the action associated with that event, and performing an action activates the associated perceptual event.{{cite book |last=Prinz |first=W. |year=1984 |chapter=Modes of linkage between perception and action |editor-first=W. |editor-last=Prinz |editor2-first=A.-F. |editor2-last=Sanders |title=Cognition and motor processes |pages=185–193 |location=New York |publisher=Springer |isbn=0-387-12855-7 }}

The idea of direct perception-action links originates in the work of the American psychologist William James and more recently, American neurophysiologist and Nobel prize winner Roger Sperry. Sperry argued that the perception–action cycle is the fundamental logic of the nervous system.{{cite journal | last1 = Sperry | first1 = R.W. | year = 1952 | title = Neurology and the mind-body problem | journal = American Scientist | volume = 40 | pages = 291–312 }} Perception and action processes are functionally intertwined: perception is a means to action and action is a means to perception. Indeed, the vertebrate brain has evolved for governing motor activity with the basic function to transform sensory patterns into patterns of motor coordination.

Background

The classical approach to cognition is a 'sandwich' model which assumes three stages of information processing: perception, cognition and then action. In this model, perception and action do not interact directly, instead cognitive processing is needed to convert perceptual representations into action. For example, this might require creating arbitrary linkages (mapping between sensory and motor codes).{{cite book |last=Massaro |first=D. W. |year=1990 |chapter=An information-processing analysis of perception and action |editor-first=O. |editor-last=Neumann |editor2-first=W. |editor2-last=Prinz |title=Relationships between perception and action: Current approaches |pages=133–166 |location=New York |publisher=Springer |isbn=0-387-52069-4 }}

In contrast, the common coding account claims that perception and action are directly linked by a common computational code.{{cite book |last=Prinz |first=W. |year=2005 |chapter=Experimental approaches to action |editor-first=J. |editor-last=Roessler |editor2-first=N. |editor2-last=Eilan |title=Agency and self-awareness |pages=165–187 |location=New York |publisher=Oxford University Press |isbn=0-19-924561-4 }}

This theory, put forward by Wolfgang Prinz and his colleagues from the Max Planck Institute for Human Cognitive and Brain Sciences, claims parity between perception and action. Its core assumption is that actions are coded in terms of the perceivable effects (i.e., the distal perceptual events) they should generate.{{cite journal | last1 = Prinz | first1 = W. | year = 1997 | title = Perception and action planning | journal = European Journal of Cognitive Psychology | volume = 9 | issue = 2| pages = 129–154 | doi=10.1080/713752551}} This theory also states that perception of an action should activate action representations to the degree that the perceived and the represented action are similar.{{cite journal | last1 = Knoblich | first1 = G. | last2 = Flach | first2 = R. | year = 2001 | title = Predicting the effects of actions: interactions of perception and action | journal = Psychological Science | volume = 12 | issue = 6| pages = 467–472 | doi=10.1111/1467-9280.00387 | pmid = 11760133| s2cid = 9061659 }} Such a claim suggests that we represent observed, executed and imagined actions in a commensurate manner and makes specific predictions regarding the nature of action and perceptual representations. First, representations for observed and executed actions should rely on a shared neural substrate. Second, a common cognitive system predicts facilitation of action based on directly prior perception and vice versa. Third, such a system predicts interference effects when action and perception attempt to access shared representations simultaneously.

Evidence for common coding

From the year 2000 onwards, a growing number of results have been interpreted in favor of the common coding theory.

For instance, one functional MRI study demonstrated that the brain's response to the 2/3 power law of motion (i.e., which dictates a strong coupling between movement curvature and velocity) is much stronger and more widespread than to other types of motion. Compliance with this law was reflected in the activation of a large network of brain areas subserving motor production, visual motion processing, and action observation functions. These results support the common coding and the notion of similar neural coding for motion perception and production.{{cite journal | last1 = Eran Dayan | first1 = E. | last2 = Casile | first2 = A. | last3 = Levit-Binnun | first3 = N. | last4 = Giese | first4 = M.A. | last5 = Hendler | first5 = T. | last6 = Flash | first6 = T.|author6-link=Tamar Flash | year = 2007 | title = Neural representations of kinematic laws of motion: Evidence for action-perception coupling | journal = PNAS | volume = 104 | issue = 51| pages = 20582–20587 | doi=10.1073/pnas.0710033104 | pmid=18079289 | pmc=2154474| bibcode = 2007PNAS..10420582D | doi-access = free }}

One of the most direct evidence for common coding in the brain now stems from the fact that pattern classifiers that can differentiate based on brain activity whether someone has performed action A or B can also classify, above chance, whether that person heard the sound of action A or B, thereby demonstrating that action execution and perception are represented using a common code.{{Cite journal | doi = 10.1371/journal.pone.0003690| title = Testing Simulation Theory with Cross-Modal Multivariate Classification of fMRI Data| journal = PLOS ONE| volume = 3| issue = 11| pages = e3690| year = 2008| last1 = Etzel | first1 = J. A. | last2 = Gazzola | first2 = V. | last3 = Keysers | first3 = C. | pmid=18997869 | pmc=2577733| bibcode = 2008PLoSO...3.3690E| doi-access = free}}

In the early 21st century, the common coding theory received increased interest from researchers in developmental psychology,{{cite journal | last1 = Sommerville | first1 = J. A. | last2 = Decety | first2 = J. | year = 2006 | title = Weaving the fabric of social interaction: Articulating developmental psychology and cognitive neuroscience in the domain of motor cognition | journal = Psychonomic Bulletin & Review | volume = 13 | issue = 2| pages = 179–200 | doi=10.3758/bf03193831 | pmid=16892982| s2cid = 14689479 }} cognitive neuroscience,{{cite journal | last1 = Jackson | first1 = P.L. | last2 = Decety | first2 = J. | year = 2004 | title = Motor cognition: A new paradigm to investigate social interactions | doi = 10.1016/j.conb.2004.01.020 | pmid = 15082334 | journal = Current Opinion in Neurobiology | volume = 14 | issue = 2| pages = 1–5 | s2cid = 36205586 }} robotics,{{cite book |last1=Proctor |last2=Vu |name-list-style=amp |year=2006 |title=Stimulus-response compatibility: Data, theory and application |publisher=Taylor & Francis |isbn=0-415-31536-0 }} and social psychology.{{cite journal | last1 = Dijksterhuis | first1 = A. | last2 = Bargh | first2 = J.A. | year = 2001 | title = The perception-behavior expressway: automatic effects of social perception on social behavior | journal = Advances in Experimental Social Psychology | volume = 33 | pages = 1–40 |doi=10.1016/S0065-2601(01)80003-4 | isbn = 9780120152339 | hdl = 2066/207586 | hdl-access = free }}

Commensurate representation

Common coding posits, on top of separate coding, further domains of representation in which afferent and efferent information share the same format and dimensionality of representation. Common coding refers to 'late' afferent representations (referring to events in the environment) and 'early' efferent representations (referring to intended events). Such representations are commensurate since they both exhibit distal reference.{{cite journal | last1 = Prinz | first1 = W | year = 1992 | title = Why don't we perceive our brain states? | journal = European Journal of Cognitive Psychology | volume = 4 | pages = 1–20 | doi=10.1080/09541449208406240}}{{cite journal | last1 = Hommel | first1 = B. | last2 = Müsseler | first2 = J. | last3 = Aschersleben | first3 = G. | last4 = Prinz | first4 = W. | year = 2001 | title = The theory of event coding (TEC): A framework for perception and action planning | journal = Behavioral and Brain Sciences | volume = 24 | issue = 5| pages = 849–878 | doi=10.1017/s0140525x01000103 | pmid=12239891}} They permit creating linkages between perception and action that do not rely on arbitrary mappings. Common coding conceives action planning in terms of operations that determine intended future events from given current events (matching between event codes and action codes). In particular perception and action may modulate each other by virtue of similarity. Unlike rule-based mapping of incommensurate codes which requires preceding acquisition of mapping rules, similarity-based matching of commensurate codes requires no such preceding rule acquisition.

Ideomotor principle

In line with the ideomotor theory of William James (1890) and Hermann Lotze (1852), the common coding theory posits that actions are represented in terms of their perceptual consequences. Actions are represented like any other events, the sole distinctive feature being that they are (or can be) generated through bodily movements. Perceivable action consequences may vary on two major dimensions: resident vs. remote effects, and 'cool' versus 'hot' outcomes (i.e., reward values associated with action outcomes).{{cite book |last1=Dickinson |first1=A. |last2=Balleine |first2=B. W. |year=2002 |chapter=The role of learning in the operation of motivational systems |editor-first=H. |editor-last=Pashler |editor2-first=R. |editor2-last=Gallistel |title=Stevens' handbook of experimental psychology |volume=3 |pages=497–533 |location=New York |publisher=John Wiley |isbn=0-471-38047-4 }}

When individuals perform actions they learn what their movements lead to (Ideomotor learning). The ideomotor theory claims that these associations can also be used in the reverse order (cf. William James, 1890 II, p. 526): When individuals perceive events of which they know (from previous learning) that they may result from certain movements, perception of these events may evoke the movements leading to them (Ideomotor control). The distinction between learning and control is equivalent to the distinction between forward and inverse computation in motor learning and control.{{cite book |author-link=Daniel Wolpert |last1=Wolpert |first1=D. |author-link2=Zoubin Ghahramani |last2=Ghahramani |first2=Z. |year=2004 |chapter=Computational motor control |editor-first=M. S. |editor-last=Gazzaniga |title=The cognitive neurosciences |edition=3rd |pages=485–494 |location=Cambridge, MA |publisher=MIT Press |isbn=0-262-07254-8 }} Ideomotor learning supports prediction and anticipation of action outcomes, given current action. Ideomotor control supports selection and control of action, given intended outcomes.

Related approaches

While most traditional approaches tend to stress the relative independence of perception and action, some theories have argued for closer links. Motor theories of speech and action perception have made a case for motor contributions to perception.{{cite book |last=Viviani |first=P. |year=2002 |chapter=Motor competence in the perception of dynamic events: A tutorial |editor-first=W. |editor-last=Prinz |editor2-first=B. |editor2-last=Hommel |title=Common mechanisms in perception and action: Attention and Performance |volume=XIX |pages=406–442 |location=Oxford |publisher=Oxford University Press |isbn=0-19-851069-1 }}{{cite journal | last1 = Liberman | first1 = A. M. | s2cid = 14469676 | year = 1982 | title = On finding that speech is special | journal = American Psychologist | volume = 37 | issue = 2| pages = 148–167 | doi=10.1037/0003-066x.37.2.148}} Close non-representational connections between perception and action have also been claimed by ecological approaches.{{cite book |last1=Fowler |first1=C. A. |last2=Turvey |first2=M. T. |year=1982 |chapter=Observational perspective and descriptive level in perceiving and acting |editor-first=W. B. |editor-last=Weimer |editor2-first=D. S. |editor2-last=Palermo |title=Cognition and the symbolic processes |volume=2 |pages=1–19 |location=Hillsdale, NJ |publisher=Lawrence Erlbaum |isbn=0-89859-066-3 }}{{cite book |last=Gibson |first=J. J. |year=1979 |title=The ecological approach to visual perception |location=Boston |publisher=Houghton Mifflin |isbn=0-395-27049-9 }}

Today common coding theory is closely related to research and theory in two intersecting fields of study: Mirror neurons systems and embodied cognition. As concerns mirror systems, common coding seems to reflect the functional logic of mirror neurons and mechanisms in the brain.{{cite journal | last1 = Rizzolatti | first1 = G. | last2 = Craighero | first2 = L. | s2cid = 1729870 | year = 2004 | title = The mirror-neuron system | journal = Annual Review of Neuroscience | volume = 27 | pages = 169–192 | doi=10.1146/annurev.neuro.27.070203.144230 | pmid=15217330}} As concerns embodied cognition, common coding is compatible with the claim that meaning is embodied, i.e. grounded in perception and action.{{cite book |last=Noë |first=A. |year=2004 |title=Action in Perception |publisher=MIT Press |isbn=0-262-14088-8 }}{{cite journal | last1 = Barsalou | first1 = L. W. | year = 2008 | title = Grounded cognition | journal = Annual Review of Psychology | volume = 59 | pages = 617–645 | doi=10.1146/annurev.psych.59.103006.093639| pmid = 17705682 | s2cid = 22345373 }} Common coding theory has further sparked refined theoretical frameworks that build on its notion of a shared representational format for action and perception. A recent example for these refinements is the Binding and retrieval in action control (BRAC) framework.Frings, C., Hommel, B., Koch, I., Rothermund, K., Dignath, D., Giesen, C., Kiesel, A., Kunde, W., Mayr, S., Moeller, B., Möller, M., Pfister, R., & Philipp, A.: Binding and retrieval in action control (BRAC). In: Trends in Cognitive Sciences, Nr. 24, 2020, p. 375–387.

Common coding theory

Background

Evidence for common coding

Commensurate representation

Ideomotor principle

Related approaches

See also

References

Further reading