Penetration test
A penetration test is an authorised, time-boxed exercise in which testers use attacker techniques against an agreed scope in order to find weaknesses and prove they are exploitable. The proof is the point: it is what separates a penetration test from a scan, and what makes the resulting risk statement defensible.
How it works
An engagement has four parts and the first is not technical. Scoping decides what is in, what is out, what the objective is and what a tester is permitted to do, and it is written down as rules of engagement with named contacts on both sides. That document exists so that at three in the morning somebody can answer whether the traffic they are watching is us.
Then reconnaissance and enumeration: establishing what actually exists, which frequently differs from what the client believes exists. Then testing proper, where findings are confirmed by exploitation rather than by inference. Then reporting, which is the deliverable, and which is judged by whether a developer or an administrator can act on it without a follow-up call.
The methodology varies by target. A web application test works through authentication, authorisation per object, input handling, business logic and the client side. An internal test starts from a network position and works towards the directory. A cloud test looks at identity and policy before it looks at workloads. What is constant is that a finding is not reported until it has been demonstrated, and that each finding carries a reproduction.
The time box is a genuine constraint and it should be stated as one. A test says what was found in the time available, against the scope agreed, with the access provided. It does not say the system is secure, and any report that implies otherwise is overselling.
What goes wrong
The most common way an engagement produces less than it should is scope drawn around what is comfortable rather than around what is reachable. An application is in scope and the identity provider that fronts it is not. A subsidiary is excluded and has a network link to headquarters. An attacker does not honour scope, so every exclusion is a statement that a particular risk will not be measured, and it belongs in the report as one.
The second is credentials. Testing an authenticated application without accounts for each role means testing the login page and calling it an application test. Most serious findings in modern applications are authorisation failures, and none of them are reachable without at least two accounts at each level.
The third is the retest. A report is a list of things that were true on a date. Without a retest, nobody knows whether the fix worked, and we regularly find that a remediation closed the exact parameter reported and left the same flaw on three others. A retest that only verifies the reported instance is a weaker product than one that checks whether the class was addressed.
The fourth is treating the report as the outcome. The outcome is the change. A report that ranks findings without saying which single change closes the most of them is technically correct and operationally useless, and it is a large part of why penetration testing has a reputation for producing documents rather than improvements.
Penetration test, vulnerability assessment and red team
Two comparisons decide most purchasing decisions in this market, and both are worth setting out side by side. The first is against a vulnerability assessment: these are sold as similar products at different prices and they answer different questions.
| Vulnerability assessment | Penetration test | |
|---|---|---|
| Method | Automated scanning, mostly | Manual testing with tools as support |
| Question answered | What is known to be missing or outdated | What can an attacker actually do |
| Findings | Everything the scanner recognises | What was confirmed by exploitation |
| False positives | Common, need triage | Eliminated by proving each finding |
| Business logic flaws | Not found | The main thing manual testing adds |
| Chained attack paths | Not found | Usually the report’s headline |
| Cadence | Continuous or monthly | Periodic, and after significant change |
The two are complements, not alternatives. Continuous scanning as part of vulnerability management keeps the known-and-patchable under control. Penetration testing finds what a scanner cannot recognise: broken authorisation, logic that can be abused, and chains where three unremarkable issues combine. An organisation that only does the second is testing an estate whose basic hygiene it never measured.
The second comparison is against a red team, and the answer changes what is bought.
| Penetration test | Red team | |
|---|---|---|
| Objective | Find as many real weaknesses as possible in scope | Achieve a specific goal without being stopped |
| Coverage | Broad, deliberately | Narrow, whatever route works |
| Defenders know | Usually yes | Usually no, beyond a small trusted group |
| Detection is measured | Rarely | Always, it is the point |
| Best for | Improving a system’s security | Testing whether the organisation notices |
| Prerequisite | None beyond scope | A defence capability worth testing |
The prerequisite in the last row is the one clients most often skip. Running a red team against an organisation with no detection capability produces an expensive confirmation of something already known. Penetration testing first, then detection engineering, then adversary emulation is the order that produces value in each step.
Common mistakes
Buying a scan and calling it a test. If nothing was proved, nothing was tested.
Excluding production. Test environments differ from production in configuration, data and integrations, which are precisely the things that produce findings.
Testing once a year regardless of change. A test after a major release tells you about that release. A calendar date tells you about the calendar.
Judging a report by its length. The useful measures are whether each finding has a reproduction, whether severity is justified in this context, and whether the recommendations are specific to the system rather than generic.
Not fixing the class. Remediating the reported instance and not the pattern guarantees the same finding next year with a different parameter.
How to get more out of one
Prepare. Provide accounts for every role, documentation for the API, a test environment that mirrors production, and a named technical contact who can answer a question within the day. The proportion of an engagement spent on access problems is time not spent finding anything.
Scope by risk rather than by comfort, and where something must be excluded, record why in the report so the residual risk is visible to whoever reads it later.
Ask for the attack path, not only the finding list. The most valuable page in an internal report is usually the one showing the route from an ordinary user to the objective, because it identifies the single step whose removal breaks the whole chain.
Agree the retest up front, and agree that it covers the class of issue rather than the exact parameter. Then feed what was found back into where it came from: a class of finding that recurs is a process problem, and it is cheaper to fix in the process than in the code.
Where a regulation requires the testing, say so at scoping. The ENS, ISO 27001 and DORA all expect evidence of a specific kind, and shaping the engagement to produce it costs nothing if it is decided at the start and is expensive afterwards.
Where this shows up in an audit
The report is the audit evidence, and auditors ask consistent questions of it: what was in scope, when it was done, who did it, what was found, what severity, and what happened next. A report that answers the first five and not the sixth leaves the client to assemble the remediation record separately, so we structure findings to make that record straightforward.
Each finding carries a reproduction, the observed impact in this environment, and a recommendation specific to the system tested. Severity is argued from what we reached, not asserted from a table, and where a score is quoted we say what it is based on. We do not publish a house severity distribution, because it would be a number without a population behind it.
Different scopes need different engagements, and the hands-on assessment of a customer-facing application is the most commonly requested of them, usually because it is the part of the estate that faces customers.
FAQ
What is the difference between a penetration test and a vulnerability scan? A scan reports what it recognises from a database of known issues, with false positives to triage. A penetration test confirms exploitability, finds logic and authorisation flaws that no scanner recognises, and chains issues together. They answer different questions and most organisations need both.
How often should we run a penetration test? After significant change, and at a cadence your obligations require. Annually is common for compliance; it is a poor fit for a system that releases weekly, where testing tied to major releases plus continuous scanning between them is the more honest arrangement.
Does a penetration test guarantee we are secure? No, and any report claiming that is wrong. It states what was found in the time available, against the scope agreed, with the access provided. Change any of those three and the result changes.
Should we do a penetration test or a red team? A penetration test if you want to find and fix weaknesses broadly. A red team if you already have detection and response worth measuring and you want to know whether it works. Doing the second before the first usually produces a costly restatement of what you already knew, and a purple team exercise is often the better middle step.