The role of expressive audio-visual information on cross-modal identity matching

Supervisor: Dr. Elisa Pellegrino

Audiovisual cues are vital in communication but the processes underpinning the integration of the two modalities in speaker recognition are only partially understood. In this study series, we will investigate qualities of expressive audiovisual speech information on individuals’ ability to predict voice from face and face from voice (henceforth cross-modal identity matching).We will collect an audiovisual corpus of expressive and less expressive registers (e.g. infant- and adult-directed speech) and will process the recordings to isolate dynamic facial and voice information. dentity matching performance scores will be collected by presenting these stimuli to participants online. Results will shed light on the role of expressive audible and visible speech gestures in communication.

• Praat
• Online experiment builder