Talks and presentations


Slices of Attention in Asynchronous Video Job Interviews

September 04, 2019

Poster Presentation, 8th International Conference on Affective Computing & Intelligent Interaction (ACII 2019), Cambridge, United Kingdom

The impact of non verbal behaviour in a hiring decision remains an open question. Investigating this question is important, as it could provide a better understanding on how to train candidates for job interviews and make recruiters be aware of influential non verbal behaviour. This research has recently been accelerated due to the development of tools for the automatic analysis of social signals (facial expression detection, speech processing, etc), and the emergence of machine learning methods. However, these studies are still mainly based on hand engineered features, which imposes a limit to the discovery of influential social signals. On the other side, deep learning methods are a promising tool to discover complex patterns without the necessity of feature engineering. In this paper, we focus on studying influential non verbal social signals in asynchronous job video interviews that are discovered by deep learning methods. We use a previously published deep learning system that aims at inferring the hirability of a candidate with regard to a sequence of interview questions. One particularity of this system is the use of attention mechanisms, which aim at identifying the relevant parts of an answer. Thus, information at a fine-grained temporal level could be extracted using global (at the interview level) annotations on hirability. While most of the deep learning systems use attention mechanisms to offer a quick visualization of slices when a rise of attention occurs, we perform an in-depth analysis to understand what happens during these moments. First, we propose a methodology to automatically extract slices where there is a rise of attention (attention slices). Second, we study the content of attention slices by comparing them with randomly sampled slices. Finally, we show that they bear significantly more information for hirability than randomly sampled slices, and that such information is related to visual cues associated with anxiety and turn taking.

TLDR : Attention Mechanism applied on facial expressions detect effectively key moment of job interview. These moments mainly occurs at the beginning and end of the answers.

Download link

HireNet: a Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews

January 29, 2019

Poster Presentation, The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) , Honolulu, Hawaii, USA

New technologies drastically change recruitment techniques. Some research projects aim at designing interactive systems that help candidates practice job interviews. Other studies aim at the automatic detection of social signals (e.g. smile, turn of speech, etc…) in videos of job interviews. These studies are limited with respect to the number of interviews they process, but also by the fact that they only analyze simulated job interviews (e.g. students pretending to apply for a fake position). Asynchronous video interviewing tools have become mature products on the human resources market, and thus, a popular step in the recruitment process. As part of a project to help recruiters, we collected a corpus of more than 7000 candidates having asynchronous video job interviews for real positions and recording videos of themselves answering a set of questions. We propose a new hierarchical attention model called HireNet that aims at predicting the hirability of the candidates as evaluated by recruiters. In HireNet, an interview is considered as a sequence of questions and answers contain- ing salient socials signals. Two contextual sources of information are modeled in HireNet: the words contained in the question and in the job position. Our model achieves better F1-scores than previous approaches for each modality (verbal content, audio and video). Results from early and late multimodal fusion suggest that more sophisticated fusion schemes are needed to improve on the monomodal results. Finally, some examples of moments captured by the attention mechanisms suggest our model could potentially be used to help finding key moments in an asynchronous job interview.

TLDR : Hierarchical and Contextual (questions and job titles) are important to model. Attention mechanism could probably inform on key moments of the job interview.

Download link


Attention Slices dans les Entretiens d’Embauche Vidéo Différés

June 05, 2020

Oral Presentation, Workshop sur les “Affects, Compagnons Artificiels et Interactions” (WACAI 2020), Ile d' Olérons, France

Nous nous intéressons à l’étude de signaux influents dans les entretiens vidéo d’embauche asynchrones découverts par des méthodes d’apprentissage profond. Le système que nous étudions emploie des mécanismes d’attention, qui permettent d’extraire d’un entretien les informations et les instants décisifs (qui ont influencé la décision du système au niveau de l’entretien), sans requérir d’annotation locale. Alors que la majorité des approches similaires évaluent les mécanismes d’attention en se contentant de visualiser les moments d’attention maximale, nous proposons ici une méthodologie permettant d’automatiser l’analyse du contenu de ces attention slices afin de fournir des éléments d’interprétation des prédictions du système.

TLDR : Les tranches de vidéos selectionnées par le mécanisme d’attention ont une meilleure prédictabilité que des tranches aléatoires. Ces tranches ont lieu le plus souvent au début et en fin d’interactions pour les modalités audio et vidéo.

Lien de téléchargement

Entretien vidéo différé : modèle prédictif pour pré-sélection de candidats sur la base du contenu verbal

June 14, 2018

Oral Presentation, Workshop sur les “Affects, Compagnons Artificiels et Interactions” (WACAI2018), Ile de Porquerolles, France

Les entretiens vidéo différés sont devenus plus en plus populaires dans le milieu des ressources humaines et constituent un objet de recherche en traitement informatique de signaux sociaux. Dans cet article, nous passons en revue plusieurs études sur les recherches pertinentes en termes d’analyses comportementales et de systèmes d’entraînement à passer des entretiens d’embauche. Nous présentons ensuite un premier modèle de prédiction automatiques des classements des candidats pour un recruteur. Nous nous focalisons dans cette première étude sur l’analyse automatique du contenu verbal d’entretiens vidéo différés. Ce travail s’inscrit en partenariat avec la société EASYRECRUE, société proposant une plateforme d’entretiens vidéo différés, avec laquelle un corpus de plus de 300 candidats évalués par un recruteur a été collecté. Nos travaux ont montré la faisabilité de l’exploitation du contenu verbal après l’utilisation d’un outil de reconnaissance automatique. Les perspectives en ce qui concerne le traitement automatique de la prosodie et des comportements non verbaux sont présentées.