LANS Seminar

Seminar Title: LLMs as Black Boxes

Speaker: Carlo Graziani, Computational Scientist, Mathematics and Computer Science Division, Argonne National Laboratory

Date: Thursday, May 7, 2026

Time: 2:30 PM-3:30 PM (In-Person)

Location: Hybrid, Bldg. 240, Conference Room 4301

Description: I will discuss some work on Large Language Models (LLMs) informed by a model-agnostic view of such models, wherein one diverts attention away from what such models are (attention, feed-forward, encoding, decoding, etc.) and focuses instead on what they do:  Learn an approximation to the distribution of some data, and concomitantly learn to optimize a decision-choosing rule in some decision space.  This outlook situates LLMs very naturally in the mathematical framework of stochastic process theory: a trained LLM is in fact a stochastic process indexed over natural numbers, and valued in discrete, finite sets (the token vocabularies). I will illustrate the possibilities inherent in this outlook by exhibiting a new tool for Explainable AI: A Prompt Token Probabilistic Attribution Score, which reflects the importance of each token in a prompt to the observed LLM response.  The score is constructed by reconstructing the LLM’s approximation to the distribution over its training texts, using the LLM’s emitted log-probabilities and a bit of Bayesianism.  I will show the results of some numerical experiments, in which use of the attribution score permits interesting explanations of some odd LLM behavior in terms of the quality of the training and the nature of the training data.  At the end of the talk, I will discuss some current work on using the model-agnostic outlook to understand why the pre-training/fine-tuning workflow is successful in conferring subject matter expertise on an LLM, and why persona-prompting works to elicit better model output.