Bug: session_usage estimation fallback sums whole-session history into output_tokens (empty-model phantom rows, skews Analytics)

[SavadiA8L]

Summary

session_usage accumulates phantom output_tokens records with an empty model field. When a run falls back to the estimation path (provider did not return real usage) and the session has no compression snapshot, the estimator sums the token count of the entire session's assistant/tool history and writes that total as a single row's output_tokens. This produces records where a tiny input maps to a multi-million-token output, with model, provider, source, agent all empty and api_calls = 0.

This does not affect real billing or real cache behaviour, but it heavily pollutes the Analytics / Usage dashboard: total output tokens and the effective cache-hit ratio are both skewed, because these rows carry huge output_tokens with cache_read_tokens = 0.

Environment

• hermes-web-ui: v0.6.30 (also reproduced across local builds back to early July; the accounting function is byte-identical in every version)
• DB: hermes-web-ui.db, table session_usage

Observed data

Aggregate over all rows where model = '' (and source/agent/provider empty, api_calls = 0):

rows	input_tokens	output_tokens	cache_read	api_calls
3626	11,258,704	318,712,269	0	0

Representative individual rows (session ids redacted):

input_tokens	output_tokens	api_calls	cache_read	model
14	2,755,307	0	0	(empty)
44	1,307,782	0	0	(empty)
34	920,135	0	0	(empty)

Same session, consecutive rows monotonically increasing (each turn re-sums the whole history):

input_tokens	output_tokens
430	855,126
445	855,513
465	866,477
475	866,799
491	867,227
509	877,080

A 14-token input producing 2.75M output with api_calls = 0 is physically impossible for a real call — it is the full session history summed into one row.

Root cause

packages/server/src/services/hermes/run-chat/usage.ts

estimateUsageTokensFromMessages() sums all assistant/tool messages in the session:

const outputTokens = messages
  .filter(m => m.role === 'assistant' || m.role === 'tool')
  .reduce((sum, m) => sum + countTokens(contentToUsageText(m.content)) + countTokens(String(m.tool_calls || '')), 0)

calcAndUpdateUsage() only slices to new messages when a compression snapshot exists:

const estimateSnapshotUsage = (messages) => {
  if (snapshot && messages.length && snapshot.lastMessageIndex >= 0 && snapshot.lastMessageIndex < messages.length) {
    const newMessages = messages.slice(snapshot.lastMessageIndex + 1)   // correct: only new messages
    const newUsage = estimateUsageTokensFromMessages(newMessages)
    return { inputTokens: countTokens(SUMMARY_PREFIX + snapshot.summary) + newUsage.inputTokens, outputTokens: newUsage.outputTokens }
  }
  return estimateUsageTokensFromMessages(messages)   // BUG: no snapshot -> whole-history sum
}

So a session without a compression snapshot that hits the estimation fallback re-sums its entire output history on every turn.

Additionally, db/hermes/usage-store.ts updateUsage() does INSERT OR IGNORE with no upper-bound / sanity check on output_tokens, so the inflated value lands unfiltered.

Suggested fix

1. In the no-snapshot branch, also scope the output estimate to messages produced by the current turn/run, rather than the full session history (mirror what the snapshot branch already does).
2. Optionally, guard at write time in updateUsage(): reject or flag estimation rows where output_tokens grossly exceeds input_tokens and provider/model are empty.

Relation to prior issues

• Analytics 使用量低估：cache_read 与 api_server 聚合口径不完整 #407 (Analytics usage under-reporting / api_server aggregation gap) is a different problem — that is under-counting; this is estimator over-counting. Opposite direction, different code path.
• Bug: session_usage 表缺少 created_at 列导致旧版数据库迁移失败 #1087 also touches session_usage schema but is unrelated (missing created_at column).