Which is better, an on-premises or a cloud AI SOC?
Neither is better in general. A cloud AI SOC is quicker to start and asks little of your team, but your alerts, log excerpts and the AI’s reasoning about them are processed by someone else, somewhere else. An on-premises AI SOC keeps that data inside your network and makes the running cost of each investigation small and predictable, but you need suitable hardware and someone to look after it.
The choice usually comes down to three questions. Are you allowed to send alert data outside? How many alerts do you expect, now and in a year? And do you have people who can run a server with a graphics processor in it? If the first answer is no, the decision is mostly made. If it is yes, the other two decide it.
This guide goes through what actually leaves the network in a cloud deployment, how speed and cost compare, what the hardware looks like in plain terms, who maintains what, and how air-gapped and hybrid set-ups work. It ends with a checklist you can take into a supplier meeting. Our separate guide on on-premises AI for security covers why alert data is so sensitive; this one is about making the choice.
What leaves your network in a cloud AI SOC
Suppliers often say a cloud AI SOC only “sees alerts”. In practice an AI investigator needs much more than the alert to reach a verdict, and each piece it reads has to travel to wherever the model runs. It is worth listing the flows one by one, because each has its own owner, retention period and risk.
The biggest flow is not the alert itself but the context gathered around it. To judge a suspicious PowerShell alert, the investigator pulls the process tree, the command lines of nearby processes, the user’s recent sign-ins and other alerts on the same machine. Those excerpts carry host names, internal addresses, user names and file paths. The prompts sent to the model, and the model’s replies, hold all of this together with the AI’s view of what is weak in your environment.
Ask each supplier for this list in writing, with where each item is stored, for how long, and who can read it, including their own support staff and any sub-processors.
- Alerts: the rule name, severity, host, user and the matching event
- Log excerpts: process trees, command lines, sign-in records, network connections pulled in during the investigation
- Prompts and replies: the text sent to the language model and what it answers, which together describe your weaknesses
- Investigation records: the questions asked, searches run and the final verdict, often kept for reporting
- Threat-intel lookups: addresses, domains and file hashes sent to outside reputation services
- Telemetry: usage, error and performance data the product sends home, which can include fragments of the above
- Identity data: names and email addresses of the people who review verdicts and log on
How speed and cost per alert compare
Network distance is rarely what makes an AI investigation slow. Most of the time goes on the model thinking and on queries to your SIEM. What matters more is capacity. A hosted service shares its capacity between many customers and may limit how many requests you can send at once, so a sudden burst of alerts can queue at their end. On your own hardware you have no neighbours, but you also have a fixed ceiling: if the server can run a few investigations side by side, the rest wait.
There is one speed cost particular to the cloud. The investigator sits outside your network and queries your SIEM over the internet, or the SIEM sends data out to it. Each question in an investigation can mean a round trip. For an investigation that asks a dozen questions, the connection and its security checks add up, and a slow or filtered link to your SIEM can stall the whole thing.
Cost follows a different shape in each model. Hosted AI is usually charged by the amount of text processed, so cost rises with each alert and each extra question the investigator asks. Long investigations on busy machines cost more than short ones. On-premises, most of the cost is fixed: the server, its power and the time of the people who look after it. After that, each extra investigation costs very little. At low or uneven volumes, paying per use is often cheaper. At steady, high volumes, owning the hardware tends to win.
Whichever you consider, ask for the cost of one investigation of a typical alert from your own environment, not an average from a brochure, and ask how that figure changes when an investigation runs long.
What hardware an on-premises AI SOC needs
In plain terms: one server with one or more graphics processors in it, usually called a GPU server. Graphics processors are the chips that run AI models quickly. What matters most is how much memory they have, because the model and the text of the investigation it is working on must both fit. A model fine-tuned for security investigations can be much smaller than the largest general-purpose models, which is what makes one server enough for many organisations.
Beyond the processor, the server needs the ordinary things a busy server needs: enough cooling, a stable power supply, fast storage and a short network path to your SIEM. An AI investigator working through a queue overnight keeps the hardware busy for hours, so it runs hot. Pacing the work so the processor is not held at its limit is sensible, and it is better to plan for steady load than for peaks.
Size it from your own alerts. Run a sample of real alerts through the system before buying at scale, note how long each investigation takes and how many can run at once, and work out whether the queue clears in good time on a busy day. Plan for a second machine, or a clear fallback plan, if a single server failing would leave alerts waiting too long.
- A server with one or more graphics processors with enough memory for the model
- Cooling and power for long periods at high load
- Fast local storage for the model and investigation records
- A short, reliable network path to the SIEM and other data sources
- A plan for when the server fails or needs maintenance
Who maintains what in each model
A cloud AI SOC moves most of the running work to the supplier. They keep the model up to date, add capacity, patch the servers and watch for faults. Your team still owns the connections from your tools, the business context the investigator relies on, and the review of its verdicts. That last part does not go away in any model: people decide what to do about a real attack.
On-premises, the supplier still writes the software and improves the model, but your team installs updates, looks after the server and watches whether the queue keeps up. In practice this is the kind of work an infrastructure team already does for other servers. The extra skill is mostly knowing how to read the graphics processor’s health and load, and that can be learned.
Write down the split before you sign. A short table of tasks with a named owner for each avoids the common gap where both sides assume the other is watching.
- Model updates: supplier in the cloud; supplier ships, you install, on-premises
- Server hardware, power and cooling: supplier in the cloud; you on-premises
- Capacity when alert volume grows: supplier in the cloud; you on-premises
- Connections to your SIEM, EDR and ticketing: you in both
- Business context, such as what each machine does: you in both
- Reviewing verdicts and deciding what to do: you in both
Can an AI SOC run on an air-gapped network, and what about hybrid?
Yes, if everything it needs runs inside. An air-gapped network has no connection to the internet, so a cloud AI SOC is ruled out. An on-premises one can work if the model, the investigation software and the data it queries all sit on the isolated network. The things to plan for are updates and threat intelligence: model and software updates arrive on approved media through your usual transfer process, and reputation lookups use a feed you import rather than a live service. Expect intelligence to be a little older than on a connected network, and say so in verdicts that lean on it.
Hybrid set-ups sit between the two, and several shapes are common. Some organisations run the AI on-premises but use a hosted console for reporting, which keeps raw alert data inside while summaries leave. Others run sensitive business units on-premises and the rest in the cloud. A third pattern runs on-premises with a cloud model as an overflow when the local queue is long.
That third pattern needs care. A cloud fallback is convenient, but it quietly sends data out exactly when you are busiest, which is often during an incident. If data must stay inside, switch any external fallback off and size the hardware so the queue clears without it. If some data may leave, write down which data and under which conditions, and check the setting at each review.
A checklist for choosing
Use these questions to narrow the choice before talking to suppliers, and again when comparing their answers. The first two often settle the matter on their own.
If the answers are mixed, a hybrid design or a hosted pilot followed by an on-premises roll-out are both reasonable. The pilot tells you how the investigator performs on your alerts, and the move inside comes once you know the hardware you need.
- Do contracts, regulators or internal policy limit where alert data may be processed?
- Is any part of the network air-gapped or restricted from the internet?
- How many alerts a day do you expect, and how uneven is the volume?
- Do you have people who can look after a GPU server, its updates and its load?
- How quickly does a real attack on your most important systems need a person’s attention?
- Which outbound flows (lookups, telemetry, notifications) are you willing to keep?
- Can any external AI fallback be switched off, and can you check that it is off?
- What does one investigation of a typical alert from your environment cost in each model?
- Who owns each running task, and is that written down?
How armor360 approaches it
armor360 runs on its own AI model, fine-tuned by our team. It can run fully on-premises, the AI included, on your own hardware, so alert data need not leave your environment, or we can host it for you. External AI fallback can be switched off. Our estimate of the running cost is about $0.10 of compute per investigation; that is a running cost, not a price.
Whichever way it runs, armor360 investigates, gives a verdict with the evidence cited, and recommends next steps. People review the verdict and decide what to do. Pricing is by quote, sized to what you protect, with no charge per person.