Báo cáo khoa học: The Modulation of Cooperation and Emotion in Dialogue: The REC Corpus

In this paper we describe the Rovereto Emotive Corpus (REC) which we collected to investigate the relationship between emotion and cooperation in dialogue tasks. It is an area where still many unsolved questions are present. One of the main open issues is the annotation of the socalled “blended” emotions and their recognition. Usually, there is a low agreement among raters in annotating emotions and, surprisingly, emotion recognition is higher in a condition of modality deprivation (i. e. only acoustic or only visual modality vs. bimodal display of emotion). ...
Nội dung trích xuất từ tài liệu:
Báo cáo khoa học: "The Modulation of Cooperation and Emotion in Dialogue: The REC Corpus" The Modulation of Cooperation and Emotion in Dialogue: The REC Corpus Federica Cavicchio Mind and Brain Center/ Corso Bettini 31, 38068 Rovereto (Tn) Italy federica.cavicchio@unitn.it Caldognetto et al., 2004) and gesture (Poggi, Abstract 2007). Moreover, great efforts have been done to analyze multimodal aspects of irony, persuasion In this paper we describe the Rovereto Emotive or motivation. Corpus (REC) which we collected to investigate Multimodal coding schemes are mainly focused the relationship between emotion and coopera- on dialogue acts, topic segmentation and the so tion in dialogue tasks. It is an area where still called “emotional area”. The collection of mul- many unsolved questions are present. One of the timodal data has raised the question of coding main open issues is the annotation of the so- called “blended” emotions and their recognition. scheme reliability. The aim of testing coding Usually, there is a low agreement among raters scheme reliability is to assess whether a scheme in annotating emotions and, surprisingly, emo- is able to capture observable reality and allows tion recognition is higher in a condition of mod- some generalizations. From mid Nineties, the ality deprivation (i. e. only acoustic or only visu- kappa statistic has begun to be applied to vali- al modality vs. bimodal display of emotion). Be- date coding scheme reliability. Basically, the cause of these previous results, we collected a kappa statistic is a statistical method to assess corpus in which “emotive” tokens are pointed agreement among a group of observers. Kappa out during the recordings by psychophysiologi- has been used to validate some multimodal cod- cal indexes (ElectroCardioGram, and Galvanic ing schemes too. However, up to now many mul- Skin Conductance). From the output values of these indexes a general recognition of each emo- timodal coding schemes have a very low kappa tion arousal is allowed. After this selection we score (Carletta, 2007, Douglas-Cowie et al., will annotate emotive interactions with our mul- 2005; Pianesi et al., 2005, Reidsma et al., 2008). timodal annotation scheme, performing a kappa This could be due to the nature of multimodal statistic on annotation results to validate our data. In fact, annotation of mental and emotional coding scheme. In the near future, a logistic re- states of mind is a very demanding task. The low gression on annotated data will be performed to annotation agreement which affects multimodal find out correlations between cooperation and corpora validation could also be due to the nature negative emotions. A final step will be an fMRI of the kappa statistics. In fact, the assumption experiment on emotion recognition of blended underlining the use of kappa as reliability meas- emotions from face displays. ure is that coding scheme categories are mutually1 Introduction exclusive and equally distinct one another. This is clearly difficult to be obtained in multimodal In the last years many multimodal corpora have corpora annotation, as communication channelsbeen collected. These corpora have been recorded (i.e. voice, face movements, gestures and post-in several languages and have being elicited with ure) are deeply interconnected one another.different methodologies: acted (such as for emo- To overcome these limits we are collecting ation corpora, see for example Goeleven, 2008), new corpus, Rovereto Emotive Corpus (REC), atask oriented corpora, multiparty dialogs, corpora task oriented corpus with psychophysiologicalelicited with scripts or storytelling and ecological data registered and aligned with audiovisual da-corpora. Among the goals of collection and analy- ta. In our opinion this corpus will allow to clear-sis of corpora there is ...