***M.Sc. Students, these seminars can count towards the six mandatory Telfer Research Seminars Series required for your program (MGT 6191/ MGT 6991 / MHS 6991) (4 seminars for MSc Project-based students).***
Warut Khern-am-nuai, PhD
Large language models are increasingly used in groups that discuss a problem and then vote or produce a joint recommendation. Group deliberation is often expected to make AI decisions more reliable. We find that it can instead make them less predictable. In a fully deterministic, self-hosted benchmark, exact reruns are identical, but small wording changes intended to preserve a scenario’s meaning can grow over repeated discussion and often change the final recommendation. In deployed black-box API systems, nominally identical committee runs also vary at temperature 0, a setting many users expect to be nearly deterministic. Across 12 policy scenarios, the results show that instability is not only a consequence of deployment-side variation: repeated interaction can make a committee sensitive to small differences in how a case is presented. Additional deployed experiments show that role structure, model composition, and the amount of prior discussion agents can revisit all change the degree of divergence. Collective AI therefore presents a stability problem as well as an accuracy problem. Deterministic execution alone does not ensure that a committee’s recommendation is predictable or readily auditable.
About the Speaker
Warut Khern-am-nuai is the FRQ–IVADO Chair in Economics and Ethics of Artificial Intelligence and Professor of Information Systems at the Desautels Faculty of Management, McGill University. His research interests include the economics and ethics of artificial intelligence, platform for online marketplaces, and management of information security. He earned his Ph.D. in Management Information Systems from the Krannert School of Management, Purdue University, in 2016.
