I built an LLM API abuse detector. The benchmark kept lying to me.
What broke when I tried to tell credential misuse from legitimate AI traffic without reading prompts or responses.
I started with a fairly ordinary question: if somebody gets hold of an LLM API key, what can the provider actually see?
A provider sees requests, timestamps, model names, token counts, errors, and the source directly connected to it. It usually does not see the original user behind a gateway. It does not know whether two people sharing a key have permission to do so. And I did not want a system that inspected prompts or responses to make up for that.
I thought I could build a useful detector from metadata. Over a few iterations, I found several ways to fool myself instead.
What a stolen key looks like
I began with a controlled OpenAI-compatible endpoint and nonfunctional canary credentials. The first capture was small: 55 requests from seven source addresses. Callers discovered models, sent liveness checks, tried aliases, and changed request formats.
Some of that looked obviously programmatic. That did not make it malicious. A legitimate coding agent can query models, retry failures, test aliases, run continuously from cloud infrastructure, and switch providers after an error. A launch, batch job, or incident recovery can create the same rate and concurrency changes as a stolen key.
The honeypot gave me behavior shapes. It could not give me a production false-positive rate. Anything that reaches a honeypot was selected because somebody found it.
The gateway problem
I tested the same workload with curl, Python clients, OpenAI SDKs, LangChain, and Node fetch—first directly, then through LiteLLM and Bifrost.
Directly, the clients were distinguishable through stable application-layer fingerprints: user agents, SDK headers, header order, and other nonvolatile fields. Through either gateway, those differences disappeared. Upstream saw the gateway's own application fingerprint. The same held for the last TLS hop: a relay replaced the original client's TLS stack with its own.
This is not a gateway bug. A gateway receives one request and creates another; it does not forward the original TCP connection or TLS session.
The consequence is uncomfortable: a legitimate platform and an unauthorized relay can have the same last-hop topology. A provider often cannot infer every client or user behind a noncooperating gateway. So the question changed from what kind of client sent this request? to is this credential still being used like the workload to which it was issued?
The simulator gave me a great result. It was wrong.
My first serious simulator generated credential-level feature rows directly: rate change, source novelty, model novelty, and so on. The detector appeared to catch more than 90% of one simulated relay family.
That was conceptual leakage. The generator created a distributed relay by assigning the feature changes that a distributed relay was supposed to have. The detector then learned those same changes and appeared to rediscover the scenario. There was no literal label column, but I had still designed the world around the model's inputs.
I threw that result out.
The replacement simulator generates raw provider-visible request events first. A credential has a base rate, token distribution, model mix, error rate, timezone, infrastructure, creation time, registered gateways, and declared deployments. A scenario changes the request-generating process—not a feature row. A separate, label-blind pipeline then derives features from the events and customer context.
The hard limit: metadata cannot observe permission
I made 100 exact authorized/unauthorized pairs. Each pair has identical request times, counts, token consumption, model selection, source infrastructure, errors, customer context, and derived features. One credential is labeled as an authorized shared operation. The other is labeled as unauthorized pooling.
If a model scores the two members differently, identity or truth information has leaked into the pipeline. The current pipeline gives them identical scores.
That is not a limitation to tune away. If authorization is the only difference and authorization is not observable, no metadata classifier can separate the two cases.
I also got hourly recall of 100%. That was wrong too.
My first hourly evaluator reported that it detected all 168 simulated incidents. The bug was in the definition of detection: it gave an incident credit whenever an already-open review case overlapped the incident. A case could open before an incident, remain active, and get credit when the incident began without any new threshold crossing.
Once I required a new case opening during the incident, the hourly result fell from 168 detections to three.
That changed the unit of evaluation. The question is not “how many suspicious rows overlapped an incident?” It is “how many operational cases opened during an incident?” A useful detector should create a review at a time when someone can do something with it.
Where the work stands
The current experiment uses five temporal views and creates cases rather than firing an alert for every high-scoring row. On the primary synthetic policy, the system opened 264 cases; 222 overlapped modeled unauthorized activity and 42 were authorized cases. The policy missed its own false-case budget.
Those numbers are evidence that the model ranks patterns inside this simulator. They are not production validation, probabilities of compromise, or a false-positive rate for real customers.
Behavior should not be the first thing a production system checks. Stronger evidence comes first: credential state, revocation, canary or exposure records, proof of possession, workload identity, registered gateways, signed delegation, and reconciliation between an application session, gateway request, provider invocation, and billed usage.
That is the system I am building toward at InferTrail: not an oracle that declares fraud, but a privacy-preserving way to investigate unexplained AI usage, runaway agents, and possible credential misuse without reading prompts or responses.