AI & LLM
Prompt injection, jailbreaks, training data poisoning and model extraction, plus the guardrails and red teaming that apply to LLM applications and agents.
Adversarial example
An adversarial example is an input crafted with a small, deliberate perturbation that causes a machine learning model to produce a wrong output while looking unchanged to a person. It is the classical attack on classifiers, and it is still what matters wherever a model makes a security or safety decision.
AI agent security
AI agent security is the practice of securing software that decides its own next step and acts through tools, which makes it an autonomous actor inside your systems rather than a feature. It covers the agent’s own identity and credentials, what it may reach, the integrity of its memory and tools, and a record of everything it did.
AI supply chain
In AI security, the AI supply chain is the full set of external components a model-backed system depends on: downloaded model weights, third-party datasets, embedding stores, agent tools and the libraries around them. Treating these as a supply chain, and testing them as one, is the discipline almost no product glossary applies.
Membership inference
Membership inference is an attack that determines whether a particular record was part of a model’s training set, by observing how the model responds to it. It is the question a data protection officer asks when a model has been trained on customer data, because a positive answer is a disclosure about an identifiable person.
Model Context Protocol (MCP)
In AI security, the Model Context Protocol (MCP) is an open standard that lets a language model application connect to external tools and data through a common interface. It is how an AI agent acquires the capabilities it acts with, and therefore where much of an agent’s real attack surface now lives.
Model extraction
Model extraction is an attack in which repeated queries to a model are used to build a functional copy of it. The attacker never touches the weights: they use the interface as an oracle, collecting input and output pairs until a substitute model reproduces the original’s behaviour closely enough to be useful.
Model inversion
Model inversion is an attack that reconstructs representative features of the data a model was trained on, using only access to the model’s outputs. Where the training set contained personal data, a successful inversion turns a deployed model into a disclosure of that data, which is a privacy problem before it is a security one.
Shadow AI
In security, shadow AI is the use of AI tools inside an organisation without approval or oversight: staff pasting company data into public assistants, teams wiring unsanctioned models into workflows. It is the new, uncontrolled data-leak channel, and it is what makes the older understanding of data loss prevention out of date.
System prompt leakage
System prompt leakage is the disclosure of the instructions an application places in front of a language model: its role, its rules, its tool descriptions and anything else the developer wrote there. It is an entry in the OWASP list for LLM applications, and the real finding is usually not the disclosure but what was put in the prompt.
Tool poisoning
Tool poisoning is an attack in which the description or metadata of a tool available to an AI agent is written to manipulate the model. Because the agent reads those descriptions to decide what to call and how, the description is untrusted input that reaches the model with the authority of configuration.
Training data poisoning
In AI security, training data poisoning is the manipulation of the data used to train or fine-tune a model so it behaves as the attacker intends: a hidden trigger, a bias, or a degraded capability. It is an attack on the model’s supply chain, and one rarely covered from the tester’s perspective.