What our AI verdict labels mean

Why the labels matter

A verdict is only useful if the person reading it knows exactly what it claims. This post sets out the five labels armor360 gives an investigated alert, what each one means, why we keep authorised activity apart from harmless, why we would rather say unsure than guess, and what a reviewer should do with each.

The labels are deliberately few. A long scale of threat levels invites arguments about the difference between level six and level seven. Five labels, each with a clear meaning and a clear next step, are easier to act on and easier to check. Each label comes with a confidence, a written reason and the records the agent collected, so the label is never the whole answer.

• Real attack: the evidence shows malicious activity.
• Suspicious: something is wrong-looking and needs a person, but the evidence does not prove an attack.
• Harmless: the behaviour is benign and explained.
• Authorised activity: the behaviour would be worrying from anyone else, but someone allowed to do it did it.
• Unsure: the available data cannot settle the question.

Real attack and suspicious

Real attack means the agent found evidence that an adversary, or malicious software, is behind the alert. In our lab, an alert called “Burst of reconnaissance commands” on a Linux machine called linux04 was judged a real attack at 90% confidence. The agent had asked 13 questions, read 4 data sources and kept 22 records, and among them it found a process named caldera-agent calling out over HTTP every five minutes. That is the kind of finding a real attack verdict should rest on: something concrete a person can look at.

Suspicious sits one step below. The behaviour does not fit the normal pattern and the agent cannot explain it away, but nothing proves an attack either. A typical example is an interactive logon to a database server from a workstation that has never connected to it before, by an account that is valid and has the right permissions. It could be lateral movement. It could be a new team member. The agent says what it saw and what it could not rule out.

For a reviewer, real attack means read it now, check the cited evidence and pass the recommended next steps to whoever can act. Suspicious means apply what you know about the company. Often one fact the logs do not hold, such as a project starting this week, settles it either way.

Why authorised is separate from harmless

Harmless means the activity itself is benign. A package manager writing to system folders during an update, a monitoring agent reading process lists, a browser downloading a signed installer from the vendor’s own site: these trip rules but carry no risk in themselves.

Authorised activity is different. The activity is exactly what an attacker would do, and it would be a real concern if a stranger did it. It is acceptable only because of who did it and why. An administrator dumping the list of domain users, a penetration tester running a credential-access tool, an engineer opening a remote shell on a production server during a change window: none of these is harmless in the abstract.

Keeping the two apart matters for three reasons. First, authorised activity depends on the person still being allowed to do it; if the administrator’s account is later stolen, the same commands become an attack, and the label reminds a reviewer that permission was the deciding fact. Second, auditors and risk teams often want to know how much powerful activity happens on sensitive systems, and a separate label lets you answer that. Third, it keeps the agent honest. Calling a domain-wide user enumeration “harmless” would teach everyone to stop looking at it.

For a reviewer, both labels mean the alert is set apart from the working queue, and both deserve a regular sample rather than a full read. With harmless verdicts, check that the reason really explains the behaviour, not just that the process name looks familiar. With authorised activity, check that the permission is still current: that the person still holds the role, the change window is still open and the penetration test has not ended. When an authorised pattern turns up outside the hours or machines it normally covers, change the verdict and say why.

Why an honest unsure beats a forced guess

Any system that must always pick between attack and not-attack will sometimes guess. A guess dressed as a verdict is worse than no verdict, because it carries the same authority as a judgement backed by evidence. A reviewer who sees “harmless, 55%” may well move on.

armor360 returns unsure when the data cannot settle the question, and it says what is missing. Common causes are practical: the machine stopped sending logs before the alert, the process chain is broken because the parent was not recorded, or the alert concerns an account that nobody has described. An unsure verdict with the gap named is useful work. It tells a reviewer exactly which question to ask, and sometimes it points to a coverage problem worth fixing so that the next alert on that machine can be settled.

An unsure verdict is also a signal about your environment. If the same machine keeps producing unsure verdicts because its process events are missing, the fix is in log collection, not in the AI. Fixing it improves every later investigation on that machine.

For a reviewer, an unsure verdict means decide whether the gap is worth closing. Ask the machine’s owner, check a source the agent cannot reach, or record what you know so that the next investigation has it.

How to read confidence alongside the label

Each verdict carries a confidence percentage. It is the agent’s own statement of how strongly the evidence it collected supports the label. It is not a probability measured over thousands of past alerts, and it should not be read as one. It is best used as a guide to how much checking a verdict deserves.

A high-confidence real attack with concrete evidence, like the beaconing process in our lab, needs a quick check that the cited records say what the reason claims, then action from a person. A lower-confidence real attack deserves a closer read of the evidence, because the agent is telling you it found something but not enough to be firm. A high-confidence harmless verdict on a well-known pattern needs little time. A lower-confidence harmless verdict is one of the ones worth sampling.

• Read the label, then the confidence, then the reason, then the evidence.
• Check that the records cited actually support the reason given.
• Treat a low confidence as a request for a closer look, not as a weaker version of the label.
• When you disagree, change the verdict and say why; that is remembered for later investigations about your company.

Where armor360 fits

armor360 investigates alerts from Elastic Security and its other connected tools, and gives each one of these five labels with a confidence, a reason and the cited records. A person reviews each verdict and agrees with it or changes it, and those corrections are kept for later investigations about the same company.