False positive

Any detection can be judged by crossing two questions: what the tool said, and what was actually happening. That gives four boxes, and only two of them are correct.

True positive: it alerted and something was there. True negative: it stayed quiet and nothing was there. Those two are the job done properly, although only the first one gets noticed.

False positive: it alerted and nothing was there. False negative: it stayed quiet and something was. Both are errors and they are not interchangeable: a false positive costs time, a false negative costs the whole incident.

And here is the knot, because both errors are tied to the same dial. Lowering a rule’s threshold so as to miss nothing drives false positives up; raising it for a quiet life produces false negatives. There is no setting that removes both, so the work is not eliminating them but deciding deliberately which of the two you prefer on each rule, and why.

Rules that describe shape rather than intent. A signature looking for a string or a traffic pattern does not know what it was being used for. The same remote administration tool is routine in the hands of the systems team and a problem in anyone else’s.

Every organisation’s own environment. A rule that behaves well in one place fills another with alerts, because here there is a nightly process copying files in bulk and there only an attacker does that. Out-of-the-box rules do not know your house, and nobody tunes them for you.

Outdated signatures and configuration. A rule written against an old version of an application keeps firing once the application changes behaviour for entirely legitimate reasons.

Scanners that infer rather than verify. Many analysis tools flag a vulnerability because the version number the server advertises appears on a list, without checking whether the flaw is genuinely exploitable there. That is fair as a warning and it is not a finding, and confusing the two is what fills reports.

One false alert costs the time it takes to dismiss it. A steady stream of them costs something else and much worse: the team learns that opening alerts does not pay. That is called alert fatigue, and it is not an attitude problem. It is the reasonable response to a system that is usually wrong.

The damage is invisible while it happens. It shows up afterwards, when an incident review finds that the signal was there, that it fired in time, and that it was buried among two hundred others that were nothing. A SOC drowning in noise is not failing for lack of tooling.

There is a second cost that gets discussed less: false positives distort priorities. If a third of the vulnerability list is not real, the order in which things get fixed stops meaning anything, and vulnerability management becomes spreadsheet management.

Here the term shifts slightly and the difference is worth drawing out. A scanner can be right about what it sees and wrong about what it concludes: the version is what it says, the CVE is real, and the flaw still cannot be used on that system because the affected component is not in play or something in front of it prevents the path.

That is not a defect of the scanner, it is its limit. An automated tool measures potential exposure; only somebody who tries can say whether that exposure is real. It is why CVSS describes the theoretical severity of a flaw rather than what it means on your network, and why EPSS adds the likelihood of exploitation but still knows nothing about how your house is built.

There is a reverse that gets named less often and hurts more: the scanner’s false negative. Business logic flaws and issues that only matter when chained have no signature and no version number, so no tool marks them. A report with zero findings produced by a tool does not say there is nothing there. It says the tool saw nothing.

Tune against what is normal here. Before writing exceptions you have to know what normal looks like in this organisation: which processes run at night, which administrator uses which tool, which traffic climbs on Mondays. Without that baseline, every exception is a hole opened blind.

Write rules with context, not only with signatures. That is what detection engineering is about: treating rules as code, with a hypothesis, tests and review, instead of accumulating inherited alerts nobody dares to touch.

Automate the dismissal, not the judgement. A SOAR platform can enrich an alert and close on its own the ones that meet a checkable condition. What does not get automated is deciding what deserves a look.

Measure. How many alerts are closed as false positives, which five rules generate the most, and how much time goes into them. Without that number the conversation runs on impressions, and the noisy rule survives because nobody knows which one it is.

And go looking rather than waiting. Threat hunting starts from a hypothesis rather than an alarm, so it finds precisely what the rules do not cover. It is the natural counterweight to the false negative.

An intrusion detection system is configured to recognise traffic patterns associated with a denial of service. On a Monday morning the alert fires: volume far above normal against a single service.

The team starts responding. Reviewing the source shows the traffic coming from expected ranges and lining up with an email campaign marketing launched that morning. There was no attack. There were customers.

The alert was a false positive, but the fault was not in the rule so much as in what the rule did not know. Nobody had told it that a Monday like this could exist. The fix is not to disable it either, because then the real attack would go unseen as well: the fix is to give it context and, incidentally, to have the next campaign announced before it goes out. This example is illustrative and describes no client.

In our web pentesting the rule is that whatever goes into the report was achieved, not inferred. Every finding carries the steps to reproduce it and the evidence of what was obtained, and anything that could not be demonstrated is labelled for what it is: an observation, with that word in front of it.

It is more work than exporting a tool’s list, and it is the difference between a document that organises a year of work and one that buries it. It is also why the retest is part of the service: checking the fix with the same test that found the flaw is the only thing that closes the matter.