3.1.3 Attribution of Model Behavior to Data
Original Problem in the Paper
Paper motivation: if training data causes undesirable downstream behavior, governance needs attribution from behavior/properties back to data points, data composition, pretraining, fine-tuning/preference data, and synthetic data. Dedicated open problems: understand pretraining-data effects on behavior; understand properties/effects of preference data; understand synthetic-data impacts on performance/bias; balance tractability and accuracy in attribution methods such as influence functions/TRAK, which had not scaled to largest foundation-model settings.
July 2026 Update & Trajectory
Substantial research progress makes attribution more tractable at research scales: TRAK, EK-FAC influence functions up to 52B, controlled data-mixture benchmarks, and empirical work on synthetic-data/model-collapse risks. Still not solved for governance-grade attribution in frontier models: closed data/weights block access, causal attribution across pretraining/RLHF/synthetic data remains uncertain, and methods can be scale- or approximation-limited. No verified 2026 primary source showed robust attribution for frontier closed models.
Deployed / Operationalized
- Influence functions using EK-FAC have been applied to LLMs up to 52B parameters to study which training examples contribute to behaviors and generalization patterns.
- TRAK is released as code and demonstrated across ImageNet classifiers, CLIP, BERT, and mT5, reducing the tractability/efficacy trade-off relative to retraining thousands of models.
- DataComp-LM operationalizes controlled data-curation experiments and downstream evaluation at 412M-7B scale.
- Synthetic-data risk assessment is operationalized in papers showing recursive training can cause model collapse and in best-practice guidance emphasizing factuality, fidelity, diversity, and bias checks.
New Tractable Vectors
- Approximate influence analysis for large-but-not-frontier LLMs and candidate filtering/query batching for attribution workflows.
- Controlled measurement of how data filtering, deduplication, and mixtures affect 7B-class model performance.
- Evaluation of recursive/synthetic-data training regimes and preservation of original human data to mitigate collapse.
- Preference-data quality questions, such as diversity/representativeness/aggregation, can be isolated in fine-tuning studies even if end-to-end attribution remains hard.
Key Open Questions
- Causal attribution of specific harmful behaviors in frontier models to exact pretraining examples, fine-tuning data, reward data, or synthetic data.
- Attribution under closed weights/data and after post-training interventions that obscure provenance.
- Governance-grade confidence intervals: when is evidence strong enough to demand dataset removal, disgorgement, or liability?
- Attribution across multimodal and agentic behaviors, not just text outputs or benchmark predictions.
- Synthetic-content provenance and quantifying model-collapse risk in live web-scale crawls after widespread AI-generated content.