Description
This tutorial aims to introduce basic tools of human behavior analysis to students of both computer science and psychology, to enable collaborations between these disciplines. We will cover different application areas, discuss examples to illustrate the possibilities, as well as challenges and pitfalls of methodology.
Program
The half-day program is planned as follows:
- Part 1: Introduction to multimodal behavioural analytics and interactive AI. Issues in experiment design, including data collection, feature extraction and selection, machine learning pipeline, in-the-wild data, bias, overlearning.
- Part 2: Face analysis: FACS coding, automatic analysis of face image and videos, example applications. Annotation and data quality. Affect and emotion estimation. Off the shelf tools (OpenFace, PyAFAR).
- Part 3: Body Dynamics: Body and pose estimation. Action and activity recognition. Wearable sensors. Temporal models. Off the shelf tools (OpenPose and more recent tools).
- Part 4: Speech and paralinguistic analysis. Multimodality. Off the shelf tools (OpenSmile). Interaction analysis, including synchrony, rapport, mimicry, speed, intensity, regularity, extent.
- Part 5: Multimodal data fusion and recent development in vision-langugage models for multimodal behavioural analytics.
- Part 6: Presentations on example human-robot/computer interaction research projects by lecturers. Discussion and presentation (optional) by students about their own related projects.
Target audience
The course is suitable for both computer science and psychology students, as well as students of other programs in related areas. It is at the introductory level, its coverage is broad, rather than deep. Because it covers different modalities (i.e. face, body, speech), it may be of interest to students who focused on a single modality so far in their research. Reading materials will be made available to the participants. No programming expertise is required; the course is suitable for both computer science and psychology students, as well as students of other programs in related areas.
Contact
For any questions, please contact A.A. Salah (a.a.salah at uu.nl).
