Model extraction
Model extraction is an attack in which repeated queries to a model are used to build a functional copy of it. The attacker never touches the weights: they use the interface as an oracle, collecting input and output pairs until a substitute model reproduces the original’s behaviour closely enough to be useful.
How well it works depends on what the interface gives back. An endpoint that returns confidence scores or full probability distributions leaks far more per query than one that returns a single label, and one that returns explanations leaks more still. Query selection matters too: probing near the decision boundary is worth many times a random sample, which is why volume alone is a poor detector.
Two consequences follow, and the second is the one that is usually missed. The direct loss is commercial, in that the training investment is reproduced for the cost of the queries. The indirect loss is security: a local copy lets an attacker develop and test attacks offline, against a stand-in, and then apply what works to the real system without generating the failed attempts a defender would have seen.
The controls are unglamorous and they work in combination: authenticate every caller, apply rate limiting per identity rather than per address, return the least information the use case needs, and monitor for query patterns that sweep a space rather than solve a problem. We do not publish figures on how many queries a given model requires, because that number depends entirely on the model and the interface. Establishing it for a specific deployment, alongside model inversion exposure, is what the AI testing where the model itself is treated as the asset sets out to do.