Language is a compositional system that can express an unlimited number of different messages by recombining a finite list of elements. This property characterizes all human language, whether spoken or signed. Where might this pervasive property of language have originated in our species? Our hypothesis is that compositionality might transcend language and could be rooted in a more "primitive" human communication system: the physical expression of emotions. We are specifically interested in the involuntary, spontaneous use of emotional expressions, in contrast with emotional signals that are intentionally meant to communicate. We build on a large body of research that has examined face and body displays in emotion and aim to determine whether compositionality characterizes these displays.
We have launched the project by collecting 350 athletes’ pictures, moments after they won or lost a high stakes competition. We annotated their facial expressions using FACS (Ekman, Friesen & Hager, 2002) and body postures using BACS (Marom & Cavicchio 2015, see attached file). We found clusters of prototypical features for win and for loss, respectively, and some that overlap. Our initial results show that a compositional approach to understanding corporeal displays of emotion is crucial for investigating emotion (see paper presented at GESPIN 2015). We also find that athletes often display combinations of features, and understanding how these complex combinations are interpreted will be the key to understanding the compositional system. We are now conducting experiments to determine how participants categorize the emotions conveyed by different combinations of features in the same spontaneously occurring displays.
Publications:
Cavicchio, Federica, Dachkovsky, Svetlana, Leemor, Livnat, Shamay-Tsoory, Simone, and Sandler, Wendy. (2018). Compositionality in the language of emotion. PloS ONE 13(8): e0201970.
Cavicchio, Federica and Sandler, Wendy. (2015). Towards a theory of compositionality in displays of extreme emotion. Proceedings of GESPIN.
The GRAMBY body arrangement coding system. (2015).
Presentations:
Cavicchio, Federica, Leemor, Livnat, Shamay-Tsoory, Simone, Dachkovsky, Svetlana and Sandler, Wendy. (2017). Compositionality in the expression of emotions. ISCOP & GRAMBY Workshop.
Leemor, Livnat, Cavicchio, Federica, Shamay-Tsoory, Simone and Sandler, Wendy. (2017). The winner takes it all: Victory elicits greater empathy than defeat. ISCOP & GRAMBY Workshop.
Research team:
Researcher: Federica Cavicchio
Researcher: Livnat Leemor
Senior Researcher: Simone Shamay-Tsoory
Principal Investigator: Wendy Sandler
