Award Abstract # 2028876
Collaborative Research: PPoSS: Planning: Hardware-accelerated Trustworthy Deep Neural Network

NSF Org: CCF
Division of Computing and Communication Foundations
Recipient: RUTGERS, THE STATE UNIVERSITY
Initial Amendment Date: August 7, 2020
Latest Amendment Date: August 7, 2020
Award Number: 2028876
Award Instrument: Standard Grant
Program Manager: Danella Zhao
dzhao@nsf.gov
 (703)292-4434
CCF
 Division of Computing and Communication Foundations
CSE
 Direct For Computer & Info Scie & Enginr
Start Date: October 1, 2020
End Date: September 30, 2022 (Estimated)
Total Intended Award Amount: $70,000.00
Total Awarded Amount to Date: $70,000.00
Funds Obligated to Date: FY 2020 = $70,000.00
History of Investigator:
  • Yingying Chen (Principal Investigator)
    yingche@scarletmail.rutgers.edu
Recipient Sponsored Research Office: Rutgers University New Brunswick
3 RUTGERS PLZ
NEW BRUNSWICK
NJ  US  08901-8559
(848)932-0150
Sponsor Congressional District: 12
Primary Place of Performance: Rutgers University New Brunswick
NJ  US  08854-3925
Primary Place of Performance
Congressional District:
06
Unique Entity Identifier (UEI): M1LVPE5GLSD9
Parent UEI:
NSF Program(s): PPoSS-PP of Scalable Systems
Primary Program Source: 01002021DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 026Z
Program Element Code(s): 042Y
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Deep-learning approaches have recently achieved much higher accuracy than traditional machine-learning approaches in various applications (e.g., computer vision, virtual/augmented reality, and natural language processing). Existing research has shown that large-scale data from various sources with high-resolution sensing or large-volume data-collection capabilities can significantly improve the performance of deep-learning approaches. However, state-of-the-art hardware and software cannot provide sufficient computing capabilities and resources to ensure accurate deep-learning performance in a timely manner when using extremely large-scale data. This project develops a scalable and robust heterogeneous system that includes a new low-cost, secure, deep-learning hardware-accelerator architecture and a suite of large-data-compatible deep-learning algorithms. It allows deep learning to fully benefit from extremely large-scale data and facilitates efficient, low-latency applications in connected vehicles, real-time mobile applications, and timely precision health. The new technologies resulting from this project can enable more research opportunities to design new hardware accelerators for deep learning and obtain further optimization in computational complexity and reduction in power consumption. Moreover, by integrating the research results with the undergraduate and graduate curricula and outreach activities, this project has great impacts on education and training of researchers and engineers for computer architecture, security, theory and algorithms, and systems.

This project designs trustworthy hardware accelerators optimized for large-scale deep-learning computations and models the complicated structure of large-scale datasets. More specifically, this project develops a novel hardware accelerator for deep learning that can achieve low power consumption. In addition, this project designs innovative in-memory encryption schemes to secure the neural models in deep-learning accelerators. Furthermore, data-modeling and statistical-learning algorithms are developed in this project to further reduce the computing cost of deep learning when processing extremely large-scale datasets. Finally, this project builds and evaluates a prototype of the proposed heterogeneous deep-learning system in terms of efficiency, scalability, and security in multiple application domains including mobile applications, connected vehicles and precision health.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Yang, Xin and Yang, Song and Liu, Jian and Wang, Chen and Chen, Yingying and Saxena, Nitesh "Enabling Finger-touch-based Mobile User Authentication via Physical Vibrations on IoT Devices" IEEE Transactions on Mobile Computing , 2021 https://doi.org/10.1109/TMC.2021.3057083 Citation Details
Xie, Y. and Jiang, R. and Guo, X. and Wang, Y. and Cheng, J. and Chen, Y. "mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave" International Conference on Computer Communications and Networks (ICCCN) , 2022 https://doi.org/10.1109/ICCCN54977.2022.9868878 Citation Details
Shi, Cong and Zhao, Tianming and Zhang, Wenjin and Mahdad, Ahmed Tanvir and Ye, Zhengkun and Wang, Yan and Saxena, Nitesh and Chen, Yingying "Defending against Thru-barrier Stealthy Voice Attacks via Cross-Domain Sensing on Phoneme Sounds" 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS) , 2022 https://doi.org/10.1109/icdcs54860.2022.00071 Citation Details
Hu, B. and Zhao, T. and Xie, Y. and Wang, Y. and Guo, X. and Cheng, J. and Chen, Y. "MIXP: Efficient Deep Neural Networks Pruning for Further FLOPs Compression via Neuron Bond" Proceedings of International Joint Conference on Neural Networks , 2021 Citation Details
Shi, Cong and Wang, Yan and Chen, Yingying and Saxena, Nitesh and Wang*, Chen "WearID: Low-Effort Wearable-Assisted Authentication of Voice Commands via Cross-Domain Comparison without Training" Annual Computer Security Applications Conference , 2020 https://doi.org/10.1145/3427228.3427259 Citation Details
Shi, Cong and Xu, Xiangyu and Zhang, Tianfang and Walker, Payton and Wu, Yi and Liu, Jian and Saxena, Nitesh and Chen, Yingying and Yu, Jiadi "Face-Mic: inferring live speech and speaker identity via subtle facial dynamics captured by AR/VR motion sensors" the 27th Annual International Conference on Mobile Computing and Networking , 2022 https://doi.org/10.1145/3447993.3483272 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This planning project explored designing trustworthy hardware accelerators optimized for large-scale deep-learning computations and models the complicated structure of large-scale datasets. More specifically, this project develops a novel hardware accelerator for deep learning that can achieve low power consumption. Furthermore, data-modeling and statistical-learning algorithms were developed to further reduce the computing cost of deep learning when processing extremely large-scale datasets. Finally, this project developed a prototype of the proposed heterogeneous deep-learning system in terms of efficiency, scalability, and security in multiple application domains including mobile applications, connected vehicles and precision health. Particularly, we accomplished the following aspects. (1) Global Mixture Pruning for Further Flops Compression via Neuron Bond. (2) Low-Effort Personalized Fitness Monitoring Using Millimeter Wave. (3) Defending against Thru-barrier Stealthy Voice Attacks via Cross-Domain Sensing on Phoneme Sounds.

(1) Global Mixture Pruning for Further Flops Compression via Neuron Bond. Neuron networks pruning is effective in compressing pre-trained CNNs for their deployment on low-end edge devices. However, few works have focused on reducing the computational cost of pruning and inference. We found that existing pruning methods usually remove parameters without fine-grained impact analysis, making it hard to achieve an optimal solution. We developed a global mixture pruning mechanism, aiming to effectively reduce the computational cost of CNNs while maintaining a high weight compression ratio and model accuracy. We proposed removing neuron bonds between the paths between filter and output map in the convolution layer and the paths between two fully connected layers. By pruning the neuron bond, we can skip corresponding conventional computation for generating each pixel of the output map. We also designed an influence factor to analyze the importance of neuron bonds and weights in a fine-grained way so that our approach could significantly reduce the pruning iterations by only performing a one-shot pruning when the total number of unimportant parameters reaches the compress ratio. Therefore, our approach required less retraining iterations to recover the accuracy of the network.

(2) Low-Effort Personalized Fitness Monitoring Using Millimeter Wave. There is a growing trend for people to perform workouts at home due to the global pandemic of COVID-19 and the stay-at-home policy of many countries. Since a self-designed fitness plan often lacks professional guidance to achieve ideal outcomes, it is important to have an in-home fitness monitoring system that can track the exercise process of users. We designed and implemented fitness monitoring system using a single COTS mmWave device. The proposed system integrated workout recognition, user identification, multi-user monitoring, and training effort reduction modules and makes them work together in a single system. In particular, we developed a domain adaptation framework to reduce the amount of training data collected from different domains via mitigating impacts caused by domain characteristics embedded in mmWave signals. We also developed a GAN-assisted method to achieve better user identification and workout recognition when only limited training data from the same domain is available.

(3) Defending against Thru-barrier Stealthy Voice Attacks via Cross-Domain Sensing on Phoneme Sounds. The open nature of voice input makes voice assistant (VA) systems vulnerable to various acoustic attacks (e.g., replay and voice synthesis attacks). Our study found that acoustic signals passing through the barriers generally present a unique frequency-selective effect in the vibration domain. Thus, we proposed to devise a system to capture this unique effect of barriers by leveraging low-cost, cross-domain sensing available in users’ wearables. The system replayed the audio-domain signals with the wearable’s speaker and captures the conductive vibrations caused by the audio sounds in the vibration domain via the built-in accelerometer. To improve the proposed system’s reliability, we developed a unique vibration-domain enhancement method to extract the phonemes most sensitive to the frequency-selective effect of barriers. We identified effective vibration-domain features that capture the barriers’ effects in the vibration domain.

 

The new technologies resulting from this project can enable more research opportunities to design new hardware accelerators for deep learning and obtain further optimization in computational complexity and reduction in power consumption. Moreover, by integrating the research results with the undergraduate and graduate curricula and outreach activities, this project has great impacts on education and training of researchers and engineers for computer architecture, security, theory and algorithms, and systems. Results are disseminated through scholarly publications, active outreach to the wireless and mobile industry through WINLAB's industry events and connections. The project provided students with rich mentoring and research experience in experiment design, prototyping, and data analysis. Three Ph.D. students got support from this project and gained significant results. Additional master students and eight undergraduate students also participated in the project through summer internship and independent study programs.

 

 

 

 


Last Modified: 01/29/2023
Modified by: Yingying Chen

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page