Facial Animation Generation with Applications in Educational Psychology

Abstract:
This talk will present preliminary work in facial animation generation with applications in educational psychology. In the first part, we describe two psychology studies as well as the computer vision techniques and platforms being used. Both studies investigate using conversational agents (CAs) as a way of delivering medical messages to patients. By incorporating CAs in the system, both semantic and emotional information can be delivered, which helps the patients, especially those with low heath and numerical literacy, to get a better understanding of their test results and medical instructions. Human studies were conducted to test the effectiveness of CA. The second part of this talk will discuss the details of a proposed neural network based facial animation synthesis method. By unifying both appearance-based and warping-based methods in an end-to-end training process, the proposed system was able to generate vivid facial animation with highly preserved details. In addition, we integrated this network with another audio speech processing system. We show both qualitatively and quantitatively that the proposed system achieved a higher performance than baseline methods. Finally, another two studies regarding representation learning using adversarial autoencoders, as well as infant gaze direction classification will be briefly reviewed.

Bio:
Kevin Gu is a new employee who just joined 3M CRSL last September. Kevin has a background in Electrical and Computer Engineering, with a focus on image/signal processing during Bachler and Master degree, and computer vision and deep learning in PhD degree. Before joining 3M, Kevin did a summer internship at St. Paul, where he worked with people in CRSL on QR code detection and code migration from C++ to Java on Android device.