Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models
Uncovering the hidden environmental factors causing nondeterministic outputs in LLM inference.
Even when decoding with temperature $T=0$, large language models (LLMs) can produce divergent outputs for identical inputs. Recent work by Thinking Machines Lab highlights implementation-level sources of nondeterminism, including batch-size variation, kernel non-invariance, and f...
ArXiv CS.AI
Source
Generated insight by OnAirToday AI
This research exposes that LLM non-determinism stems from underlying hardware and implementation details rather than just architectural design. Quantifying background temperature is a critical step toward ensuring scientific reproducibility and consistent performance across diverse deployment environments.
Get the day's AI news in one short email. Free.
Comments
Keep comments relevant and respectful. No links, advertising, self-promotion, or spam.