Compositionality in the Expression of Emotions
Federica Cavicchioi, Svetlana Dachkovskyii, iii, Livnat Leemorii,
Simone Shamay-Tsooryii, Wendy Sandlerii
iUniversity of Padua, iiUniversity of Haifa, iiiGordon Academic College of Education
Emotions are complex internal states that we communicate with intricate displays of face and body actions. Are these displays phylogenetically hardwired and holistic motor programs, or does each face and body feature contribute an identifiable meaning to the overall display? To address this question, we borrow a concept from linguistics – compositionality – according to which complex linguistic expressions are built up from simpler parts, each contributing an element of meaning to the whole.
We coded pictures of athletes photographed within moments of winning and losing a competition. Participants rated the pictures on a continuum between opposing emotional states, e.g., submissive/dominant, ashamed/proud, sad/happy. Multiple mixed effect regressions determined which facial and body signals predicted each emotion.
We found that single features of the face distinguish among positive emotional states like happy and proud, while body features such as shoulder asymmetry vs. collapsed upper body distinguish negative emotions with unexpected outcome (frustrated, angry) from other negative emotions related to resignation (sad, ashamed). We conclude that body and face components modulate emotion recognition, potentially providing an ancient scaffolding upon which, millions of years later, the abstract and constrained compositional system of human language could build.
