Title: 

From Democratizing Optimization with LLM to Improving LLM Performance with OR Techniques

Abstract:

Recent breakthroughs in Large Language Models (LLM) have captured public imagination and interest, while mathematical optimization remains largely underappreciated outside expert circles. In this talk, we show that LLM can finally bridge the persistent gap between optimization’s potent capabilities and its limited real-world uptake. We present the 4I framework—Insight, Interpretability, Interactivity, Improvisation—as a set of design principles for combining LLM with mathematical optimization. Insight establishes a trusted, up-to-date view of the state; Interpretability explains model logic and trade-offs; Interactivity enables conversational what-if analysis; and Improvisation supports event-driven re-optimization.

We also demonstrate how OR techniques can optimize LLM inference under memory constraints. We formulate LLM inference as a multi-stage online scheduling problem with stochastic arrivals and dynamic resource consumption. We develop a fluid dynamics approximation that provides a tractable performance benchmark and guides algorithm design. Building on this foundation, we introduce two algorithms. The Waiting for Accumulated Inference Threshold (WAIT) algorithm optimizes scheduling when output lengths are known using dynamically maintained thresholds. For the realistic scenario where output lengths are unknown at arrival, the Nested WAIT algorithm adaptively learns prompt characteristics through a hierarchical multi-segment framework. We establish theoretical near optimality guarantees under heavy traffic conditions, balancing throughput, latency, and Time to First Token (TTFT). Experiments using Llama-7B on A100 GPUs demonstrate 15-30% throughput improvements and reduced latency versus industry baselines (vLLM and Sarathi).

Bio:

David Simchi-Levi holds the MIT William Barton Rogers Professorship (named after the founder & first president of MIT), is a Professor of Engineering Systems at MIT and serves as the head of the MIT Data Science Lab. He is considered one of the premier thought leaders in supply chain management and business analytics. His Ph.D. students have accepted faculty positions in leading academic institutes including U. of California Berkeley, Carnegie Mellon U., Columbia U., Cornell U., Duke U., Georgia Tech, Harvard U., U. of Illinois Urbana-Champaign, U. of Michigan, Purdue U. and Virginia Tech.

Professor Simchi-Levi is the former Editor-in-Chief of Management Science (2018-2023), one of the two flagship journals of INFORMS. He served as the Editor-in-Chief for Operations Research (2006-2012), the other flagship journal of INFORMS and for Naval Research Logistics (2003-2005). In 2023, he was elected a member of the National Academy of Engineering. In 2020, he was awarded the prestigious INFORMS Impact Prize for playing a leading role in developing and disseminating a new highly impactful paradigm for the identification and mitigation of risks in global supply chains. He is an INFORMS Fellow and MSOM Distinguished Fellow and the recipient of the 2020 INFORMS Koopman Award given to an outstanding publication in military operations research; Ford Motor Company 2015 Engineering Excellence Award; 2014 INFORMS Daniel H. Wagner Prize for Excellence in Operations Research Practice; 2014 INFORMS Revenue Management and Pricing Section Practice Award; and 2009 INFORMS Revenue Management and Pricing Section Prize.

He was the founder of LogicTools which provided software solutions and professional services for supply chain optimization. LogicTools became part of IBM in 2009. In 2012 he co-founded OPS Rules, an operations analytics consulting company. The company became part of Accenture in 2016. In 2014, he co-founded Opalytics, a cloud analytics platform company focusing on operations and supply chain decisions. The company became part of the Accenture Applied Intelligence in 2018.