October 14th, 2021 
Daniela Lopez de Luise ;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;text-decoration:none”>
Master Internship Position: Deep Learning architectures for generating skeleton-based human motion
Interested in Deep Learning and human motion analysis ? Apply to our Master internship position in deep generative models for skeleton-based human motion.
As part of the ANR DELEGATION project, you will have the opportunity to continue with a funded PhD in our team !
More information below or at : https://maxime-devanne.com/delegation/jobs/internship_position_delegation_2022.pdf
*******************************
Context:
Human motion analysis is crucial for studying people and understanding howthey behave, communicate and interact with real world environments. Dueto the complex nature of body movements as well as the high cost of motioncapture systems, acquisition of human motion is not straightforward and thusconstraints data production. Hopefully, recent approaches estimating humanposes from videos offer new opportunities to analyze skeleton-based human mo-tion. While skeleton-based human motion analysis has been extensivelystudied for behavior understanding like action recognition, some efforts are yetto be done for the task of human motion generation. Particularly, the automaticgeneration of motion sequences is beneficial for rapidly increasing the amountof data and improving Deep Learning-based analysis algorithms.
Since several years, new image generation paradigms have been possiblethanks to the appearance of Generative Adversarial Networks (GAN) which have proved to be extremely efficient for many image generation tasks and hu-man posture estimation. Although these networks are very efficient, theirexplainability and control still remain challenging tasks. Differently, other gen-erative models have also emerged by considering the data distribution duringtraining like Variational AutoEncoder (VAE) and Flow-based networks.However, when it comes to human motion, many challenges remain to be solved,in particular when passing from the static case to the dynamic case. Firstwork addressing deep generative models for human motion have considered mo-tion capture (mocap) data allowing to accurately extract body parts positionsalong the time. Hence, aforementioned generative architectures have been suc-cessively employed for generating mocap-based human motion sequences. Differently, we consider noisy skeleton data estimated from videos as it iseasily applicable in real-world scenarios for the general public.
Goal of the project:
The goal of this internship is to provide guidelines in building deep genera-tive models for skeleton-based human motion sequences. Inspiring from recenteffective Deep Learning-based approaches, the aim is to gener-ate full skeleton-based motion sequences without access to successive poses asprior information as it can be done in prediction tasks. It is therefore crucialto investigate how deep generative models can handle such noisy and possiblyincomplete data in order to generate novel motion sequences as natural andvariable as possible
In particular, the candidate will work on the following tasks:
– Deep Learning architectures for skeleton-based human motion: investigation and assessment of the influence of different deep network ar- chitectures for capturing complex human motion features. Particularly, the goal of this task is to theoretically and empirically analyze the per- formance of existing architectures like CNN, RNN and GCN for modeling skeleton-based human motion.
– Deep generative models adapted to skeleton data: based on stud- ies from the previous task, the goal is to build generative models upon the previously identified meaningful spaces where skeleton sequences are represented. Therefore, the candidate will investigate different generative models, like GAN, VAE and Flow-based models, in order to propose and develop a complete Deep Learning model for generating skeleton-based human motions.
– Evaluation of deep generative models: in order to validate the pro- posed model, experimental evaluation is crucial. In comparison to motion recognition where classification accuracy is a natural way to assess an ap- proach, evaluating the task of motion generation is not as straightforward. Dedicated metrics evaluating both naturalness and diversity of generated sequences as well as the impact of new generated sequences in a classifi- cation task will be considered.
Prerequisites:
The candidate must fit the following requirements:
– Registered in Master 2 or last year of Engineering School (or equivalent) in Computer Science
– Advanced skills in Python programming are mandatory
– Good skills in Machine Learning & Deep Learning using related
libraries (scikit-learn, Tensorflow, Pytorch, etc.) are required
– Knowledge and/or a first experience in human motion analysis will be appreciated
Research environment:
The proposed internship will be carried out within the MSD (Modeling and Data Science) team from the IRIMAS Institute. It will be part of the ANR DELEGATION project starting in 2022 for 4 years. Hence, there is a great opportunity to continue with a PhD in our team on the same topic/project.
Application:
For further information or for applying, candidates should send a CV, academic records, personal projects (e.g. github repo) and a motivation letter to maxime.devanne@uha.fr.
Posted in