Heuristic analysis

Heuristic analysis is an evaluation technique that uses general rules and approximation to identify possible threats or malicious behaviour on a system.

Unlike the traditional approach, which matches a file against a signature for a specific known piece of malware, heuristic analysis looks at behaviour and at suspicious characteristics, which means it can flag something even when no signature for it exists yet.

That is what makes it useful against new malware variants and against attacks written specifically to evade signature matching, which is most of them.

It can be applied at several levels, from evaluating an individual file or program to watching the behaviour of a whole system while it runs.

What it looks for

Anomalous behaviour. Anything that departs from the established norm: unusual commands, modification of critical files, or unexpected communication with external servers.

Unusual resource use. CPU and memory consumption that does not fit what the process should be doing, which is often the first visible sign of something like cryptojacking.

File and registry manipulation. Creation, modification or deletion of system files and registry entries that nothing legitimate should be touching.

Communication patterns. Network traffic that does not match the usual shape: connections to suspicious addresses, beaconing at regular intervals, or protocols nothing on this host should be speaking.

A worked example

An endpoint product uses heuristic analysis against a new piece of malware for which no signature exists.

During analysis it observes a program making repeated attempts to encrypt system files and opening network connections to addresses with no reputation.

The specific malware is unknown, but the behaviour is not: the product raises an alert on the pattern, quarantines the process and notifies the user.

What made this possible is that nothing in the decision depended on having seen this exact file before.

The trade-off, stated plainly

Heuristic analysis buys coverage of the unknown and pays for it in false positives, and that is not a defect to be tuned away, it is the shape of the technique.

A signature is a statement about a specific artefact and it is either right or wrong. A heuristic is a statement about resemblance, and legitimate software regularly resembles malicious software: installers write to system directories, backup agents read every file, and administrative tooling does by design what an attacker does by intent.

Which is why the same idea, taken further, ends up as endpoint detection and response: record the behaviour, alert on the parts that matter, and keep enough context that a human can decide quickly whether the alert was worth raising. And it is why detection engineering treats a detection rule as software with an owner, because a heuristic that nobody tunes becomes either noise everyone ignores or a rule so narrow it catches nothing.

It is also the reason fileless techniques matter: with no file to sign, behavioural analysis is not one option among several, it is the only one left.