SIEM

A SIEM does four things in order, and the order is why deployments fail in the middle.

It collects. Logs arrive from firewalls, identity providers, endpoints, mail gateways, cloud control planes, databases and applications, each in its own format. It normalises: field names are mapped onto a common schema so that a user name is the same field whether it came from a directory or a web server, which is the only way a query can cross sources. It correlates: rules and queries run over the normalised stream, joining events that mean nothing alone. And it retains, which is both the investigative capability and, in most regulated sectors, the reason it was purchased.

The value is entirely in the joins. A failed login is noise. A failed login from a country the account has never used, followed by a success, followed by a mailbox rule that forwards externally, followed by a download from a document store, is an incident, and no single source sees the whole sequence. That is the query a SIEM exists to answer.

Detection content is written, not bought. Vendors ship a starter set, but the rules that catch what happens to you are the ones somebody wrote against your estate: your naming conventions, your service accounts, your normal. That work is detection engineering, and it is a continuing role rather than a phase of a project.

The most common state we find is a SIEM that has everything except the source we need. Ingest is priced by volume, volume is dominated by things nobody investigates, and so the budget is spent on firewall accept logs while the identity provider’s sign-in log, which is where every modern intrusion is visible, is sampled or absent. On engagements this is decisive: our activity in the client’s identity plane frequently left a perfect trail in a log that was not being collected.

The second is the rule set nobody owns. Detection content ages. A rule written against a naming convention breaks when the convention changes; a rule that depended on a field breaks when the source is upgraded and renames it. A rule that is broken does not raise an error, it raises nothing, and nothing is indistinguishable from quiet. We have tested estates with several hundred rules where the number that had fired in the previous ninety days was in single figures, and nobody had noticed.

The third is that the SIEM records the intrusion perfectly and nobody reads it. This is not a technology problem and it is the single most frequent finding we write in this area. The console shows the sequence, correctly correlated, with the right severity, and the alert sat unopened because the analyst on shift had four hundred others.

From an attacker’s point of view, none of this requires evasion. Blending into the volume is easier than avoiding the sensors: authenticate at a normal hour, from an address in the right country, using a protocol the account already uses. What defeats that is a baseline of normal behaviour per account, which is expensive to build and is the only thing that works.

These are usually sold together and they answer different questions. The SIEM asks what happened. SOAR asks what to do about it, and does part of it without waiting for a person.

The order matters. Automating a response on top of detections you do not trust industrialises your false positives, and an automated containment action that fires wrongly on a production server teaches an organisation to switch automation off. Build the detection quality first, automate the enrichment steps second, and automate containment last and narrowly.

Buying by ingest volume rather than by question. Start from the questions you must be able to answer in an incident, work back to the sources that answer them, and collect those completely. A partial log of a critical source is worse than none, because it looks like coverage.

Leaving the vendor’s default rules on and calling it detection. They are written for an average estate. Yours is not average, and the rules that matter reference your naming, your service accounts and your business hours.

Sending endpoint telemetry into it wholesale. Full process telemetry is enormous and it already lives in the EDR, which is built for it. Send the alerts and the specific event types you correlate against, not everything.

No time synchronisation discipline. Correlation across sources depends on clocks. Hosts drifting by minutes make sequences unorderable, and that is discovered during the incident.

Measuring the team on alert closure. It rewards closing, which is the opposite of investigating. Measure how long a true positive took to reach a human and what happened next.

Write down the ten questions you would need to answer in the first hour of a serious incident: which accounts authenticated from where, what ran on this host, what was accessed in this repository, what changed in the cloud control plane. Then check, by running them, whether the SIEM can answer each one today. The ones it cannot are your ingest gaps, and that list is far more useful than a coverage percentage.

Test the rules the way you would test code. Fire a known technique deliberately in a controlled window and confirm the alert appears, which also tells you the end-to-end latency from action to console. Map coverage to MITRE ATT&CK so gaps are expressed as techniques rather than as impressions, and re-run the exercise after any platform upgrade, because upgrades rename fields.

Keep the retention aligned with the obligation and with the reality that intrusions are found late. Retention that ends before the average dwell time means the evidence of the start of the incident is gone before anybody looks for it.

We rarely test a SIEM as a product. We test whether it saw us. In a red team or purple team report the deliverable is a timeline with two columns: what we did and when, and what the client’s console holds for that moment. Three outcomes get written differently. Not recorded is an ingest finding, and the remediation is a source. Recorded but no rule is a detection engineering finding, and the remediation is content. Recorded, alerted, and nobody acted is an operations finding, and no product change will fix it.

Severity is written against the consequence of the blind spot, not against the platform. A missing firewall log is a low finding. An identity provider sign-in log that is not collected, in an estate whose entire perimeter is identity, is a critical one.

That comparison of our timeline against the client’s own console is a fixed deliverable of the adversary emulation work we do.

What is the difference between a SIEM and a SOC? The SIEM is the platform. The SOC is the team and the process that uses it. Buying the first without staffing the second produces a very expensive log archive.

Do we need a SIEM if we already have EDR and XDR? If you have any obligation to retain logs from systems that have no agent, or any need to correlate identity, network and application events with endpoint events, then yes. Endpoint products cover hosts; a great deal of an intrusion happens where there is no host.

Why does our SIEM produce so many alerts? Almost always because rules were enabled without tuning to the estate. The fix is unglamorous: baseline what is normal for each rule, suppress the known-good with a documented reason and an expiry, and delete rules nobody has ever acted on.

Can a SIEM detect an attacker using valid credentials? Only with behavioural baselines per account, and even then it is hard. An attacker who authenticates normally, from a plausible location, using a protocol the account already uses, generates events that look correct in every field. That is why detection cannot be the only control.