arxiv:2504.07096

OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens

Published on Apr 9

· Submitted by

liujch1998 on Apr 10

#2 Paper of the day

Upvote

Authors:

Jiacheng Liu ,

Taylor Blanton ,

Yanai Elazar ,

Sewon Min ,

YenSung Chen ,

Byron Bischoff ,

Aaron Sarnat ,

Jenna James ,

Bailey Kuehl ,

Karen Farley ,

Taira Anderson ,

Carissa Schoenick ,

Luca Soldaini

Abstract

We present OLMoTrace, the first system that traces the outputs of language models back to their full, multi-trillion-token training data in real time. OLMoTrace finds and shows verbatim matches between segments of language model output and documents in the training text corpora. Powered by an extended version of infini-gram (Liu et al., 2024), our system returns tracing results within a few seconds. OLMoTrace can help users understand the behavior of language models through the lens of their training data. We showcase how it can be used to explore fact checking, hallucination, and the creativity of language models. OLMoTrace is publicly available and fully open-source.

View arXiv page View PDF Add to collection

Community

liujch1998

Paper author Paper submitter 5 days ago

We release OLMoTrace, a tool that lets you trace the outputs of language models back to their full, multi-trillion-token training data in real time. We developed OLMoTrace to raise transparency & trust in LLMs.

On top of a standard chatbot experience, OLMoTrace highlights long pieces of LLM outputs that appear verbatim in the model’s training data, and shows the matching training documents. With OLMoTrace, you can see how LLMs may have learned to generate certain sequences of tokens. OLMoTrace is useful for fact checking ✅, understanding hallucinations 🎃, tracing LLM-generated “creative” expressions 🧑‍🎨, tracing reasoning capabilities 🧮, or just generally helping you understand why LLMs say certain things.

OLMoTrace is now available for the OLMo 2 and OLMoE family of models on Ai2 Playground. We also open-source our code so that anyone can enable OLMoTrace with their model’s training data.

Paper: https://allenai.org/papers/olmotrace
Blog: https://allenai.org/blog/olmotrace
Try OLMoTrace on Ai2 Playground: https://playground.allenai.org