Skip to content

[runtime] DanglingToolCallMiddleware treat tool call IDs as globally unique #3142

@ggnnggez

Description

@ggnnggez

Problem

DanglingToolCallMiddleware currently assumes that a tool_call_id is globally
unique across the entire message history when it normalizes tool-call
transcripts before a model call.

That assumption is too strong. OpenAI-compatible providers may reuse the same
tool-call ID string in different assistant turns. A transcript can still be
valid when each assistant tool-call turn is immediately followed by its own
matching ToolMessages.

When repeated IDs exist in history, the current middleware can turn a valid
transcript into an invalid one and trigger a provider 400:

Invalid request: an assistant message with 'tool_calls' must be followed by
tool messages responding to each 'tool_call_id'.

Historical Context

This issue was exposed while validating PR #2883, which changes the
summarization path to hide internal summarization output from the frontend
during context compression.

The summarization middleware is not the direct cause of the invalid transcript.
However, summarization makes the repeated-ID case easier to reach:

  1. Context compression replaces older history with a summary message.
  2. It preserves a recent valid tool-call transcript so the model keeps the
    necessary recent context.
  3. A later assistant turn may receive tool-call IDs from the provider/model
    adapter that reuse ID strings already present in the preserved transcript.

For example, a preserved turn may already contain web_search:11, and a later
assistant turn may also produce web_search:11.

This is a valid history shape as long as each assistant turn is paired with its
own tool results.

Current Behavior

Before middleware normalization, the history may be valid:

AIMessage tool_calls=[web_search:11, web_search:12, web_search:13]
ToolMessage web_search:11
ToolMessage web_search:12
ToolMessage web_search:13

...

AIMessage tool_calls=[web_search:9, web_search:10, web_search:11]
ToolMessage web_search:9
ToolMessage web_search:10
ToolMessage web_search:11

DanglingToolCallMiddleware collects existing tool results by
tool_call_id. With a single-value map, repeated IDs collapse to one
ToolMessage. During regrouping, the later assistant turn can lose its matching
tool result:

AIMessage tool_calls=[web_search:9, web_search:10, web_search:11]
ToolMessage web_search:9
ToolMessage web_search:10
# missing ToolMessage web_search:11

The malformed transcript is then sent to the provider, which rejects it.

Why This Matters

  • The middleware is intended to repair incomplete or non-adjacent tool-call
    transcripts before provider serialization.
  • It must not corrupt a transcript that was already valid.
  • PR fix: hide summarization LLM output from frontend during context compression #2883 relies on preserved recent messages after summarization, so this
    repeated-ID case must be handled for the summarization flow to work reliably.
  • Other paths may also contain repeated tool-call ID strings across turns, so
    the fix should live in the middleware that normalizes the transcript rather
    than relying on summarization to avoid the case.

Proposed Fix

Make DanglingToolCallMiddleware treat tool_call_id as scoped to a tool-call
occurrence/assistant turn instead of globally unique across the full message
history.

The minimal fix is to preserve all matching ToolMessages for a repeated ID and
consume them in occurrence order during transcript normalization, for example:

tool_call_id -> queue[ToolMessage]

instead of:

tool_call_id -> ToolMessage

This allows each assistant turn that reuses an ID string to keep its own matching
tool result.

Acceptance Criteria

  • DanglingToolCallMiddleware does not assume tool_call_id is globally unique
    across the full message history.
  • A valid transcript with repeated tool-call ID strings across separate
    assistant turns remains unchanged after normalization.
  • A non-adjacent transcript with repeated IDs is regrouped without dropping a
    later matching ToolMessage.
  • Existing dangling-tool-call repair behavior remains intact when a tool result
    is genuinely missing.
  • Tests cover the summarization-adjacent shape where preserved history and a
    later assistant turn reuse a tool-call ID string.

Notes

A possible defensive enhancement in the summarization path is to rewrite IDs in
preserved tool-call history while updating both assistant tool calls and matching
tool messages. That is not required for the minimal fix: repeated IDs across
assistant turns are valid input, and the normalization middleware should handle
them correctly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions