Learning Optimal Exit in Early-Exit Neural Networks

Speaker: Manjesh Hanawal

Abstract

Deep Neural Networks (DNNs) have shown impressive performance in various tasks, including image classification and natural language processing (NLP). However, their deployment on resource-constrained devices remains challenging due to energy consumption and delay overheads. Early-exit DNNs (EE-DNNs) have been proposed to address these challenges, which leverage side branches at intermediate layers for early inferences. How to decide which layer to exit is a question one needs to address in the EE-DNNs. Also, the optimal exits may depend on the specific application and need to be learned. We develop models and learning algorithms using the Multi-Arm Bandit framework to learn exit points that optimally tradeoff between latency and accuracy. We demonstrate how one can significantly reduce the inference latency in pre-trained models from NLP and Image classification applications with only a small drop in accuracy.

Bio

Manjesh K. Hanawal}received the BTech degree in ECE from NIT, Bhopal, in 2004, the M.S. degree in ECE from the Indian Institute of Science, Bangalore, India, in 2009, and the Ph.D. degree from INRIA, Sophia Antipolis and University of Avignon, France, in 2013. After two years of postdoc at Boston University, he joined Industrial Engineering and Operations Research at the Indian Institute of Technology Bombay, Mumbai, India, where he is an associate professor now. During 2004-2007 he was with CAIR, DRDO, working on various security-related projects. His research interests include communication networks, machine learning, cybersecurity, and 5G security. He is a recipient of Inspire Faculty Award from DST and the Early Career Research Award from SERB. He has received several research grants like MATRIX from SERB and Indo-French Collaborative Scientific Research Programme from CEFIPRA.