Featuring P. R. Kumar, Texas A&M University

Abstract: We address the problem of exploration versus exploitation that lies at the heart of reinforcement learning of dynamic systems. We describe the Biased Maximum Likelihood Method proposed to address this challenge. We present a comparative study of its regret performance in a variety of contexts ranging from Bandits to Markov Decision Processes to LQG systems. We also provide an account of regulation problems where there is no intrinsic conflict between exploration and exploitation, and present a historical account of results on stability, asymptotic behavior and robustness. [Joint work with Akshay Mete, Rahul Singh, Ping-Chun Hsieh, Yu-Heng Hung, Xi Liu, and Anirban Bhattacharya].

Bio: P. R. Kumar, B. Tech (1973, IIT Madras) and D.Sc. (1977, Washington Univ., St. Louis), was a faculty member in the Math Dept at University of Maryland, Baltimore County (1977-84), ECE and CSL at the University of Illinois, Urbana-Champaign (1985-2011), and has been at Texas A&M University since 2011. He has worked on problems in game theory, adaptive control, simulated annealing, machine learning, queueing networks, manufacturing systems, scheduling wafer fabrication plants, wireless networks and network information theory. His current research focus includes renewable energy, power systems, security, automated transportation, unmanned aerial vehicle traffic management, millimeter wave 5G, and cyber-physical systems. He is a member of the U.S. National Academy of Engineering, The World Academy of Sciences, and Indian National Academy of Engineering.  He was awarded an honorary doctorate by ETH, Zurich.  He received the Alexander Graham Bell Medal of IEEE, the IEEE Field Award for Control Systems, the Donald Eckman Award of the American Automatic Control Council, the Ellersick Prize of IEEE Communication Society, the Outstanding Contribution Award of ACM SIGMOBILE, the Infocom Achievement Award, the ACM SIGMOBILE Test-of-Time Paper Award, and COMSNETS Outstanding Contribution Award.  He is a Fellow of IEEE, ACM and IFAC. He is an Honorary Professor at IIT Hyderabad.