Research Spending & Results

• Maja J Mataric
• (213) 740-4520
• mataric@usc.edu

• Shrikanth Narayanan

• FY 2013=$8,000
• FY 2012=$19,420
• FY 2011=$28,756
• FY 2010=$24,788
• FY 2009=$16,000
• FY 2008=$900,000

• Ephraim P. Glinert
• (703) 292-8930
• eglinert@nsf.gov

Awardee Location

Primary Place of Performance

Abstract at Time of Award

Robotics is currently at the forefront of technologies with recognized potential for impacting quality of human life. In response to the large need for personalized one-on-one care for the growing populations of elderly individuals and those with special cognitive and social needs throughout life, great strides must be made in the domain of human-robot interaction (HRI) in order to bring robotics into such application domains in human everyday life. This interdisciplinary project identifies a specific set of HRI research questions in socially assistive robotics, the study of robotic systems capable of providing help through social rather than physical interaction. The research focus is on two key issues: (1) the role of the robot's physical embodiment in the interaction; and (2) the use of expressive embodied communication and user modeling toward personalized time-extended assistive interaction. A specific consideration is given to interactions under the challenges of socio-communicative heterogeneity and deficits. A novel assistive robot control architecture is developed, based on multi-modal perception, embodied expression and communication, and on-line user modeling, implemented in three different types of real-world socially assistive systems. To give a realistic context to the research, the experimental testing and evaluation are performed with children drawn from a typical population and a population with Autism Spectrum Disorders (ASD), a family of disorders that have already been identified as amenable to technological, and in particular robotic, intervention and therapy. A key contribution of the research lies in the unified and tightly integrated end-to-end approach that jointly studies embodiment and multimodal expressive communication, grounded in data from hypothesis-driven experiments, and the development and use of novel signal processing and user modeling methods for human-machine interaction design.

This project, by its very nature, aims to impact the large and growing population affected by Autism Spectrum Disorders. With the support of the large and unique corpus of data that will be collected and analyzed, the expected scientific impact will go well beyond novel insights toward a better understanding of the fundamentals of HRI and its relevance for diverse user populations and for socialization of children with ASD. In addition to the basic research, two complementary educational outreach programs broaden the impact of the work: (1) a novel K-12 outreach and teacher training program specifically aimed at special education through the use of robotics for teaching STEM topics; and (2) undergraduate research training and pipelined role-modeling from pre-university to undergraduate to graduate students. The education programs dovetail with the research plan, resulting in integrated activities involving all participants in the research team.

Publications Produced as a Result of this Research

David J. Feil-Seifer and Maja J. MatariÄ? "A Simon-Says Robot Providing Autonomous Imitation Feedback Using Graded Cueing" International Meeting for Autism Research (IMFAR), v. , 2012, p.

David J. Feil-Seifer and Maja J. MatariÄ? "Using Computational Models Over Distance-Based Features to Facilitate Robot Interaction with Children" Human-Robot Interaction, v.1, 2012, p.

Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee, Shrikanth Narayanan "Emotion recognition using a hierarchical binary decision tree approach" Speech Communication. Sensing Emotion and Affect - Facing Realism in Speech Processing, v.53, 2011, p.9

Matthew P. Black, Joseph Tepperman, and Shrikanth S. Narayanan "Automatic Prediction of Children's Reading Ability for High-level Literacy Assessment" IEEE Transactions on Audio, Speech and Language Processing, v.19, 2011, p.1015

Emily Mower, Maja J Mataric, and Shrikanth S. Narayanan "Framework for Automatic Human Emotion Classification Using Emotional Profiles" IEEE Transactions on Audio, Speech and Language Processing, v.19, 2011, p.1057

David J. Feil-Seifer and Maja J. MatariÄ? "Ethical Principles for Socially Assistive Robotics" IEEE Robotics and Automation Magazine, v.18, 2011, p.24

Joseph Tepperman, Sungbok Lee, Abeer Alwan, Shrikanth S. Narayanan "A generative student model for scoring word reading skills" IEEE Transactions on Audio, Speech, and Language Processing, v. , 2010, p.

David J. Feil-Seifer and Maja J. MatariÃ?? "Dry Your Eyes: Examining the Roles of Robots for Childcare Applications" IEEE Intelligent Systems, v.11, 2010, p.208

David J. Feil-Seifer, Maja J MatariÄ? "Dry your eyes: examining the roles of robots for childcare applications" IEEE Intelligent Systems, v.11, 2010, p.

Serdar Yildirim, Shrikanth S. Narayanan, Alexandros Potamianos "Detecting emotional state of a child in a conversational computer game" Computer Speech and Language, v. , 2010, p.

Emily Mower, Maja Mataric, and Shrikanth Narayanan "Human perception of audio-visual synthetic character emotion expression in the presence of ambiguous and conflicting information" IEEE Transactions on Multimedia, v. , 2009, p.

Serdar Yildirim, Shrikanth S. Narayanan "Automatic detection of disfluency boundaries in spontaneous speech of children using audio-visual information" IEEE Transactions on Audio, Speech, and Language Processing, v.17, 2009, p.2

Carlos Busso, Sungbok Lee, Shrikanth S. Narayanan "Analysis of emotionally salient aspects of fundamental frequency for emotion detection" IEEE Transactions on Audio, Speech, and Language Processing, v.17, 2009, p.582

Patti Price, Joseph Tepperman, Markus Iseli, Thao Duong, Matthew Black, Shizhen Wang, Christy Kim Boscardin, Margaret Heritage, P. David Pearson, Shrikanth Narayanan, and Abeer Alwan "Assessment of emerging reading skills in young native speakers and language learners" Speech Communication, Special issue on Spoken Language Technology for Education, v. , 2009, p.

Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette Chang, Sungbok Lee, and Shrikanth Narayanan "IEMOCAP: Interactive emotional dyadic motion capture database" Journal of Language Resources and Evaluation, v. , 2008, p.

Publications Produced as Conference Proceedings

Feil-Seifer, D;Mataric, MJ "Toward Socially Assistive Robotics for Augmenting Interventions for Children with Autism Spectrum Disorders" 11th International Symposium on Experimental Robotics (ISER), v.54, 2009, p.201 View record at Web of Science