Skip directly to content

Minimize RSR Award Detail

Research Spending & Results

Award Detail

Doing Business As Name:SUNY at Stony Brook
  • Zhaozheng Yin
  • (631) 632-2592
Award Date:01/10/2020
Estimated Total Award Amount: $ 660,640
Funds Obligated to Date: $ 786,637
  • FY 2020=$109,999
  • FY 2019=$16,000
  • FY 2018=$660,638
Start Date:08/31/2019
End Date:08/31/2022
Transaction Type:Grant
Awarding Agency Code:4900
Funding Agency Code:4900
CFDA Number:47.041
Primary Program Source:040100 NSF RESEARCH & RELATED ACTIVIT
Award Title or Description:NRI: INT: COLLAB: Manufacturing USA: Intelligent Human-Robot Collaboration for Smart Factory
Federal Award ID Number:1954548
DUNS ID:804878247
Parent DUNS ID:020657151
Program:NRI-National Robotics Initiati
Program Officer:
  • Bruce Kramer
  • (703) 292-5348

Awardee Location

Street:WEST 5510 FRK MEL LIB
City:Stony Brook
County:Stony Brook
Awardee Cong. District:01

Primary Place of Performance

Organization Name:The Research Foundation for The State University of New York
Street:Old Computer Science Bld. Room 2314
City:Stony Brook
County:Stony Brook
Cong. District:01

Abstract at Time of Award

This National Robotics Initiative (NRI) collaborative research project addresses the NSF Big Idea of Work at the Human-Technology Frontier by targeting human-robot collaboration in manufacturing. Recent advances in sensing, computational intelligence, and big data analytics have been rapidly transforming and revolutionizing the manufacturing industry towards robot-rich and digitally connected factories. However, effective, efficient and safe coordination between humans and robots on the factory floor has remained a significant challenge. To meet the need for safe and effective human-robot collaboration in manufacturing, the investigators will research an integrated set of algorithms and robotic test beds to sense, understand, predict and control the interaction of human workers and robots in collaborative manufacturing cells. It is expected that these methods will significantly improve the safety and productivity of hybrid human-robot production systems, thereby promoting their deployment in future "smart factories". To broaden the impact of this project, a partnership with Manufacturing USA Institute(s) and professional societies will be established to provide human-robot collaboration learning modules for inclusion in robotics and smart manufacturing-related curricula. These learning modules, together with annual events aimed at community college and pre-college students, and workshops for the dissemination of research results will raise public awareness and attract new entrants into the manufacturing and robotics industries, creating truly synergetic education opportunities in science, technology, engineering and mathematics, as well as accelerating the adoption of smart factory-enabling technologies. The project will address fundamental challenges in human-robot collaboration in the manufacturing environment, such as the limitation of one-to-one sensing between humans and robots, the lack of adaptive and stochastic modeling methods for reliable recognition and prediction of human actions and motions in different manufacturing scenarios, and multi-scale human-robot coordination. To address these challenges, multi-disciplinary research involving sensing, machine learning, stochastic modeling, robot path planning, and advanced manufacturing will be performed. Specific tasks include algorithm development and deployment on lab-scale and real-world test beds to: (1) sense and recognize where objects (e.g., robots, humans, parts or tools) are located and what each worker is doing; (2) predict what the next human action will be; and (3) plan and control safe and optimal robot trajectories for individualized on-the-job assistance for humans, proactively avoiding worker injury. The outcomes from the project will be evaluated on the shop-floor at the collaborating company COsorizio MAcchine Uensili (COMAU) in Michigan, and the Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing of the National Research Council of Italy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Publications Produced as a Result of this Research

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Tao, Wenjin and Leu, Ming C. and Yin, Zhaozheng "Multi-modal recognition of worker activity for human-centered intelligent manufacturing" Engineering Applications of Artificial Intelligence, v.95, 2020, p.. doi:10.1016/j.engappai.2020.103868 Citation details  

Lai, Ze-Hao and Tao, Wenjin and Leu, Ming C. and Yin, Zhaozheng "Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing" Journal of Manufacturing Systems, v.55, 2020, p.. doi:10.1016/j.jmsy.2020.02.010 Citation details  

Tao, Wenjin and Al-Amin, Md and Chen, Haodong and Leu, Ming C. and Yin, Zhaozheng and Qin, Ruwen "Real-Time Assembly Operation Recognition with Fog Computing and Transfer Learning for Human-Centered Intelligent Manufacturing" Procedia Manufacturing, v.48, 2020, p.. doi:10.1016/j.promfg.2020.05.131 Citation details  

Al-Amin, Md. and Qin, Ruwen and Tao, Wenjin and Doell, David and Lingard, Ravon and Yin, Zhaozheng and Leu, Ming C "Fusing and refining convolutional neural network models for assembly action recognition in smart manufacturing" Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, v., 2020, p.. doi:10.1177/0954406220931547 Citation details  

Chen, Haodong and Tao, Wenjin and Leu, Ming C. and Yin, Zhaozheng "Dynamic Gesture Design and Recognition for Human-robot Collaboration with Convolutional Neural Networks" Proceedings of the ASME International Symposium on Flexible Automation (ISFA), v., 2020, p.. Citation details  

Chen, Haodong and Leu, Ming C and Tao, Wenjin and Yin, Zhaozheng "Design of a Real-time Human-robot Collaboration System Using Dynamic Gestures" Proceedings of the ASME International Mechanical Engineering Congress and Exposition (IMECE), v., 2020, p.. Citation details  

Moniruzzaman, Md and Yin, Zhaozheng and He, Zhihai and Qin, Ruwen and Leu, Ming C "Action Completeness Modeling with Background Aware Networks for Weakly-Supervised Temporal Action Localization" Proceedings of the 28th ACM International Conference on Multimedia (MM ’20), v., 2020, p.. Citation details  

Tao, Wenjin and Lai, Ze-Hao and Leu, Ming C. and Yin, Zhaozheng and Qin, Ruwen "A self-aware and active-guiding training & assistant system for worker-centered intelligent manufacturing" Manufacturing Letters, v.21, 2019, p.. doi:10.1016/j.mfglet.2019.08.003 Citation details  

Al-Amin, Md. and Tao, Wenjin and Doell, David and Lingard, Ravon and Yin, Zhaozheng and Leu, Ming C. and Qin, Ruwen "Action Recognition in Manufacturing Assembly using Multimodal Sensor Fusion" Procedia Manufacturing, v.39, 2019, p.. doi:10.1016/j.promfg.2020.01.288 Citation details  

For specific questions or comments about this information including the NSF Project Outcomes Report, contact us.