6.3.1 Prevention of Model Theft
Original Problem in the Paper
Motivation: capable models are valuable theft targets; broader integration expands exfiltration attack surface; securing weights and system components prevents unauthorized access that could undermine safety and national-security governance. Open problems: adequate cybersecurity against insider/outsider threats, physical/data-center/hardware/software controls, coordination across actors, threat-vector analysis, and defenses against query/API/logit/side-channel model extraction.
July 2026 Update & Trajectory
Infrastructure protection is becoming operationalized: RAND security levels, lab safeguard standards, access control, logging, insider programs, red teaming, deception, and confidential computing. However, no primary source shows model theft is solved against top-tier insiders/nation-states, and API-level model extraction remains demonstrated for production LMs. Weight security is a mature security-program problem with unsolved high-end assurance, not a solved technical primitive.
Deployed / Operationalized
- Centralized, access-controlled weight storage; least privilege; multi-party authorization; mandatory code review; hardware authentication; SIEM/SOAR logging; honeypots/fake weights; external red teams at major labs.
- RAND benchmark security levels and threat-vector taxonomy for frontier model weights.
- Confidential computing to reduce attack surface during inference/use.
New Tractable Vectors
- Security-level benchmarking for frontier labs rather than ad hoc controls.
- Model-weight canaries/honeypots and access-monitoring analytics for high-value IP.
- API hardening against logit/projection-layer extraction by limiting outputs, monitoring query patterns, and adding response-noise/rounding where appropriate.
Key Open Questions
- Defending against sophisticated insiders and state-compromised administrators without paralyzing research workflows.
- Quantifying residual risk for nation-state model theft across cloud, endpoints, supply chain, and human targets.
- Preventing partial model extraction or capability cloning through APIs while preserving legitimate evaluation/fine-tuning access.
Evidence & Primary Sources
- RAND identifies 38 attack vectors, five security levels, and recommends weight centralization, access reduction, insider programs, defense-in-depth, red-teaming, and confidential computing. (2024-05-30): https://www.rand.org/pubs/research_reports/RRA2849-1.html
- Anthropic RSP v3.3 lists ASL security safeguards including access management, compartmentalization, artifact provenance, binary authorization, model-weight access controls, centralized logs, honeypots/fake weights, physical security, and external red teaming. (2026-05-26): https://www.anthropic.com/responsible-scaling-policy
- Carlini et al. extracted nontrivial precise information from black-box production LMs, including projection matrices of OpenAI models for low query cost. (2024-03-11): https://arxiv.org/abs/2403.06634
- MITRE ATLAS includes AI Model Access, Exfiltration via AI Inference API, Extract AI Model, and AI Intellectual Property Theft in a living threat matrix. (2026-06-30): https://atlas.mitre.org/