SOC metrics that matter, and how to measure them honestly

Why SOC metrics go wrong

Security operations produce a lot of numbers: alerts received, tickets closed, rules deployed. Many are easy to count and tell you little. Some do harm, because people work towards them. A team judged on tickets closed per hour learns to close tickets quickly, not to find attacks.

Good measures describe the outcomes people care about. Are real attacks being found? How fast? How reliably? Is anything being missed? The measures below aim at those questions. None of them comes with a benchmark here, because the right target depends on your environment, and published figures are rarely measured the same way twice.

Pick a few measures and keep them. A handful, tracked the same way for a year, tells you more than a dashboard of dozens that change every quarter.

Time to verdict

Time to verdict is the time from an alert being raised to a clear call on it: real attack, suspicious, harmless or inconclusive. It matters more than time to close, because closing can wait on other teams long after the security question has been answered.

Measure it for each alert, then look at the spread, not only the average. A low average can hide a long tail of alerts that sat for days. Split it by verdict too: the time to a real-attack verdict is the one that matters most.

Be clear about when the clock starts. Start it when the detection tool raises the alert, not when someone first opens it. Otherwise the time spent waiting in a queue disappears from the measure.

Share of alerts looked into

When more alerts arrive than people can read, some are closed unread, sampled or left to age out. That is a common way of coping with volume, but it should be measured, not hidden. The share of alerts that received a real investigation tells you how much of what your rules raise is actually being checked.

Track it by severity and by rule. A team that investigates every high-severity alert and few low-severity ones has made a choice; make sure it is the choice you meant. Low-severity alerts are often where the early stages of an attack show up.

If the share is falling, that is a capacity problem worth raising early, with evidence, before it turns into a missed attack.

Reviewer agreement

Whether verdicts come from junior analysts, senior ones or an AI investigator, someone should check a sample of them. Reviewer agreement is the share of reviewed verdicts that the reviewer left as they were. Where the reviewer changed the call, record what it changed to and why.

This is the closest thing a SOC has to a quality measure. Low agreement on a particular rule or alert type shows where context is missing or judgement is weak. Pay most attention to verdicts changed from harmless to real attack: those are the ones that would otherwise have been missed.

Keep the review honest. A reviewer who sees the original verdict first tends to agree with it. Where you can, have some reviews done blind, and rotate the reviewers.

Coverage: machines that went silent

A detection programme can only see what sends it data. Coverage measures how many of the machines you mean to watch are actually sending logs, and which log types are missing from each. It is the measure most often overlooked, because a silent machine raises no alerts at all.

Count the machines that have gone quiet over a recent period, and how long each has been silent. A server whose logs dried up weeks ago without anyone noticing is a bigger risk than most alerts in the queue. Compare the list of reporting machines against your asset inventory, not only against what reported last month.

Silence has ordinary causes: an agent that crashed, a certificate that expired, a machine retired without being taken off the list. Recording the cause each time shows which fixes are worth making permanent.

Rule noise

Rule noise is the share of each rule’s alerts that turn out to be harmless. Measured per rule, it shows exactly where tuning effort will pay off. Often a few rules account for much of the noise; the per-rule view shows whether that is true for you.

Pair noise with value. A rule that is often harmless but has also caught real attacks may be worth keeping and tuning. A rule that is always harmless and has never caught anything is a candidate for a scheduled hunt or retirement, through your normal change control.

Count noise from verdicts, not from how quickly alerts were closed. An alert closed in seconds may have been harmless, or may simply not have been read.

Measuring honestly

Every measure here can be gamed or misread. A few habits keep them honest, and they matter more than the choice of tool you measure with.

Report the measures to the people who fund the SOC in plain words: how long high-severity alerts waited for a verdict, how often reviewers changed the call, and which servers went silent and for how long. That tells a budget holder far more than a count of tickets closed. Where armor360 is in use, each verdict records how sure the agent was and when it was reached, a person agrees with it or changes it, and the coverage view lists machines that went quiet with the likely cause.

• Define each measure in writing, including when the clock starts
• Report the spread, not only the average
• Show the alerts that were not investigated, not only those that were
• Count silent machines against the inventory, not last month’s list
• Review a sample of verdicts blind, and report how often reviewers changed them
• Change definitions rarely, and say when you do