Vallarium AI Spend Audit

[MEDIUM] Route 'summarize' runs 12,000 calls on premium 'flagship-xl'

'summarize' sends 12,000 requests to premium model 'flagship-xl' ($504.13). Re-priced on 'balanced-m' the same tokens would cost $100.83.

Evidence: 12,000 calls on 'flagship-xl' for route 'summarize'; current $504.13 vs $100.83 on 'balanced-m'.

Opportunity: $403.30 ESTIMATED — assumption: route 'summarize' holds quality on 'balanced-m' — validate on an eval before switching

Recommendation: Run an eval of 'balanced-m' on this route; if quality holds, switch.

References: OWASP LLM cost/efficiency guidance

[MEDIUM] Uncached prefix on classify (~6,100 tokens)

A ~6,100-token static prefix on model 'balanced-m' was sent uncached 26,000 times. Caching it charges the prefix at the cache-read rate after the first call.

Evidence: 26,000 uncached requests share prefix pf-classify-…; minimum input 6,100 tokens; 0 cache reads.

Opportunity: $385.38 ESTIMATED — assumption: the shared ~6,100-token prefix is cacheable and warm for ~90% of the 25,999 repeat calls

Recommendation: Enable prompt caching for this system prefix (mark it as the cacheable portion of the request).

References: Prompt caching (provider docs)

[LOW] Over-long output on route 'chat' (1,200 calls)

On 'chat', 1,200 of 12,000 calls generate more than 933 output tokens (>3x the 311-token median).

Evidence: Route 'chat' median output 311 tokens; 1,200 calls exceed 933.

Opportunity: $55.20 ESTIMATED — assumption: output beyond 3x the route median is unnecessary and a max_tokens cap recovers it

Recommendation: Set a max_tokens bound sized to the task on this route.

[LOW] 3,499 exact-duplicate requests on notify

3,500 requests share an identical fingerprint; 3,499 are redundant repeats (day 2026-06-15).

Evidence: Fingerprint rf-notify-st… seen 3,500 times; 3,499 redundant calls costing $47.24.

Opportunity: $47.24 MEASURED

Recommendation: Add an idempotency key and a short-lived response cache.

[LOW] Runaway session on agent: 2,200 calls

Session sess-runaway fired 2,200 calls in 1:13:18; a healthy session runs a handful. The excess looks like an unterminated loop.

Evidence: Session sess-runaway: 2,200 calls over 1:13:18; 2,195 beyond a 5-call baseline cost $46.12.

Opportunity: $46.12 ESTIMATED — assumption: iterations beyond 5 in this session produced no additional value (a runaway loop)

Recommendation: Add a per-session call cap and loop-termination guard.

[INFO] Route 'classify' is 36.0% of spend

'classify' accounts for $541.95 of the $1,505.46 analysed — a concentration worth watching.

Evidence: $541.95 on 'classify' (36.0% of total).

[INFO] Route 'summarize' is 33.5% of spend

'summarize' accounts for $504.13 of the $1,505.46 analysed — a concentration worth watching.

Evidence: $504.13 on 'summarize' (33.5% of total).