{"681842":{"#nid":"681842","#data":{"type":"event","title":"ISyE Seminar - Ramesh Johari","body":[{"value":"\u003Cp\u003ETitle: \u0026nbsp;\u003C\/p\u003E\u003Cp\u003EWhen Does Interference Matter? \u0026nbsp;Decision-Making in Platform Experiments\u003C\/p\u003E\u003Cp\u003EAbstract:\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0026nbsp;Online platforms and marketplaces use A\/B experiments to test new features and design changes. \u0026nbsp;Due to constraints on inventory, such experiments typically lead to biased estimation of treatment effects due to the presence of *interference* between treatment and control groups; this phenomenon has been extensively studied in recent literature. \u0026nbsp;By contrast, there has been relatively little discussion of the impact of interference on *decision-making*. \u0026nbsp; In this talk, we consider a benchmark Markovian model of a capacity-constrained platform, and study the impact of interference on (1) false positive probability and (2) statistical power. \u0026nbsp;We show that for a particular class of \u0022monotone\u0022 treatments (informally, treatments where the sign of the effect does not depend on the level of available inventory), using the standard t statistic with the na\u00efve difference-in-means estimator and classical variance estimator both correctly controls the false positive probability, and generally yields *higher* statistical power than any unbiased estimation method. \u0026nbsp;We show that in principle, these effects can be undermined when treatments are not monotone.\u003C\/p\u003E\u003Cp\u003EOur results have important implications for the practical deployment of debiasing strategies for A\/B experiments. \u0026nbsp;In particular, they highlight the need for platforms to carefully define their objectives and understand the nature of their interventions when determining appropriate estimation and decision-making approaches. \u0026nbsp;Notably, when interventions are monotone, the platform may actually be worse off by pursuing a debiased decision-making approach.\u003C\/p\u003E\u003Cp\u003EJoint work with Hannah Li, Anushka Murthy, and Gabriel Weintraub.\u003C\/p\u003E\u003Cp\u003EBio:\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ERamesh Johari is a Professor at Stanford University, with a full-time appointment in the Department of Management Science and Engineering (MS\u0026amp;E), and a courtesy appointment in the Department of Electrical Engineering (EE). He is an associate director of Stanford Data Science, and co-director of the Stanford Causal Science Center. He is a member of the Operations Research group and the Social Algorithms Lab (SOAL) in MS\u0026amp;E, the Information Systems Laboratory in EE, and the Institute for Computational and Mathematical Engineering. He received an A.B. in Mathematics from Harvard, a Certificate of Advanced Study in Mathematics from Cambridge, and a Ph.D. in Electrical Engineering and Computer Science from MIT. \u0026nbsp;His current research interests include market design, causal inference, and experimentation.\u003C\/p\u003E","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EOnline platforms and marketplaces use A\/B experiments to test new features and design changes. \u0026nbsp;Due to constraints on inventory, such experiments typically lead to biased estimation of treatment effects due to the presence of *interference* between treatment and control groups; this phenomenon has been extensively studied in recent literature. \u0026nbsp;By contrast, there has been relatively little discussion of the impact of interference on *decision-making*. \u0026nbsp; In this talk, we consider a benchmark Markovian model of a capacity-constrained platform and study the impact of interference on (1) false positive probability and (2) statistical power. \u0026nbsp;We show that for a particular class of \u0022monotone\u0022 treatments (informally, treatments where the sign of the effect does not depend on the level of available inventory), using the standard t statistic with the na\u00efve difference-in-means estimator and classical variance estimator both correctly controls the false positive probability, and generally yields *higher* statistical power than any unbiased estimation method. \u0026nbsp;We show that in principle, these effects can be undermined when treatments are not monotone.\u003C\/p\u003E\u003Cp\u003EOur results have important implications for the practical deployment of debiasing strategies for A\/B experiments. \u0026nbsp;In particular, they highlight the need for platforms to carefully define their objectives and understand the nature of their interventions when determining appropriate estimation and decision-making approaches. \u0026nbsp;Notably, when interventions are monotone, the platform may actually be worse off by pursuing a debiased decision-making approach.\u003C\/p\u003E\u003Cp\u003EJoint work with Hannah Li, Anushka Murthy, and Gabriel Weintraub.\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"When Does Interference Matter?  Decision-Making in Platform Experiments"}],"uid":"36374","created_gmt":"2025-04-16 16:14:46","changed_gmt":"2025-04-16 16:18:01","author":"mwelch39","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2025-04-18T11:30:00-04:00","event_time_end":"2025-04-18T12:30:00-04:00","event_time_end_last":"2025-04-18T12:30:00-04:00","gmt_time_start":"2025-04-18 15:30:00","gmt_time_end":"2025-04-18 16:30:00","gmt_time_end_last":"2025-04-18 16:30:00","rrule":null,"timezone":"America\/New_York"},"location":" ISYE Groseclose 402","extras":[],"groups":[{"id":"1242","name":"School of Industrial and Systems Engineering (ISYE)"}],"categories":[],"keywords":[],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1795","name":"Seminar\/Lecture\/Colloquium"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"177814","name":"Postdoc"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}