Security teams are discovering that the most capable AI models cost too much to run on routine, high-volume work, and they’re finding that out after the first invoice arrives. I lead a security operations team, and I’ve brought detection costs down to roughly a dollar a day for trust and safety work.
That number surprises people, because enterprise AI is supposed to be expensive. It isn’t, if you design the work correctly. The teams burning through budget are skipping the design work that determines which cases should reach a model at all.
This is especially relevant in India. Security teams deal with an enormous volume of account-takeover attempts, fraud signals, and user reports that are often across English, Hindi, and several regional languages. CERT-In alone handled more than 29 lakh cyber incidents in 2025. You can’t send every incident to the most capable model because every case creates more data to move, log, and govern, and the resulting bill would be massive. A funnel fixes that. Deterministic rules and lightweight models take the bulk of the volume, and the expensive model only sees the cases that require it.
The funnel is the cost lever
Detection funneling isn’t new. Before LLMs, security teams built layered filters to narrow high-volume event streams down to the cases worth a human analyst’s time. The same logic applies to AI spend. The narrower and more precise the funnel feeding your models, the lower your cost per accurate outcome.
In trust and safety, most abuse is identifiable before any model runs. Deterministic, rules-based pattern detection captures a significant portion of volume upfront. Account age, email provider, behavioral signals all feed automated filters that resolve the obvious cases and narrow what remains. Only the subset that clears those filters reaches an LLM. That’s where the dollar-a-day figure comes from. A well-designed funnel keeps expensive work to a minimum.
Think of it like a home security system. You don’t need a camera monitoring everyone who walks past your house. You care once someone’s actually inside, and that’s when you bring in the higher capability response.
Tiered models, tiered cost
For cases that clear the initial funnel, a lightweight model handles the first pass. The output is structured — a malicious or benign determination at high or low confidence. High-confidence outcomes resolve automatically, while low-confidence cases escalate to a more capable model with broader context and stronger reasoning.
Only a fraction of events reach that second tier. We ran structured efficacy testing across model options and found only a 1-2% accuracy difference between lightweight and frontier models for our use cases. Frontier models cost roughly five times more per token. That math only works if the cases reaching them need that capability.
Prompt engineering matters as much as model selection. One prompt I wrote for agentic detections runs over 1,900 words, covering every scenario the agent is likely to encounter, including when to escalate and when to act autonomously. Not every case needs that depth. Some trust and safety prompts are two or three sentences, but the scope of what you’re asking an agent to handle determines the precision the prompt requires.
Context is what separates accurate AI analysis from hallucination. Give a model an abuse report and ask whether the user is abusive, and it may take the report at face value. Give it the actual artifact being reported alongside the report, and it can assess independently whether the claim holds up.
Where humans still belong
Automation handles the clear cases. It’s the ambiguous ones that need judgment that agents don’t yet have.
Distinguishing a legitimate security researcher who hosts malware samples for analysis from a malicious actor hosting the same content to target others requires discernment AI still struggles with. So does a dispute in an issue thread where the terms-of-service could be interpreted differently. These cases reach a human because the question itself requires contextual reasoning the current generation of agents can’t reliably provide. The goal is to give human reviewers the time to spend on the cases that actually need them. Instead of clicking through individual events, engineers on my team are building the systems, writing the prompts, and defining the patterns that orchestrate detection at scale.
What this means in practice
Attackers try new obfuscation techniques and we adapt prompts and models to catch them. It’s iterative work, closer in spirit to detection engineering than a one-time deployment. Prompt engineering is just another form of that: write the rule, test the output, tune when accuracy slips.
The cost question is solvable if you treat it as a design problem from the start. In India, where volume rules out sending every case to the most capable model, that starts with the funnel. How much reaches a model, in what form, and with what context determines the bill. Most teams that find AI expensive haven’t made those decisions deliberately.
Authored by Matt Coons, Senior Manager of Security Operations, GitLab
