Skip to content

[BUG] Responses API silently enables strict tool validation — optional tool parameters are always filled in #843

Description

@bdegomme

Basic checks

  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code

What's broken?

With OpenAI defaulting to the Responses API in v2, function tools silently switch to strict parameter validation, so optional tool parameters stop being optional: the model is grammar-constrained to fill in every property of the schema on every call.

The cause is a server-side default mismatch that RubyLLM doesn't compensate for:

  • Chat Completions function.strict: default false (OpenAI OpenAPI spec: "Whether to enable strict schema adherence... default: false")
  • Responses FunctionTool.strict: default true ("Whether to enforce strict parameter validation. Default true.")

Protocols::Responses::Tools.tool_for renders {type:, name:, description:, parameters:} and never sends strict, so the server default applies. On 1.x / Chat Completions that meant non-strict; on the Responses API it means strict. Under strict mode, OpenAI requires every property to be listed in required (optionality is expressed as nullable types instead), so schemas with genuinely optional parameters no longer behave as designed.

The upgrading guide covers the Responses API switch but doesn't mention this behavior change, so it's easy to hit in production without noticing — tool calls keep succeeding, just with every optional filter populated.

How to reproduce

  1. Define a tool with optional parameters:
class SearchTool < RubyLLM::Tool
  description "Searches calls with optional filters"

  parameters type: "object",
    properties: {
      keyword:   { type: "string",  description: "Optional title filter. Omit unless the user asks." },
      date_from: { type: "string",  description: "Optional ISO date filter. Omit unless the user asks." },
      page:      { type: "integer", description: "Omit unless you need the next page." }
    },
    required: [],
    additionalProperties: false

  def execute(keyword: nil, date_from: nil, page: nil)
    { keyword:, date_from:, page: }
  end
end
  1. RubyLLM.chat(model: "gpt-5.4").with_tool(SearchTool).ask("search for calls about pricing")
  2. Inspect the tool call arguments (RUBYLLM_DEBUG=true, or the on_tool_call callback).

Expected behavior

The model omits parameters it doesn't need — e.g. {"keyword": "pricing"} — as it did on Chat Completions (RubyLLM ≤ 1.x behavior for the same tool, same model).

What actually happened

Every property is populated on every call, e.g. {"keyword": "pricing", "date_from": "2026-01-01", "page": 1} — the model invents values for filters it was told to omit, changing search results. Same tool + same model on protocol: :chat_completions behaves as expected, confirming it's the strict default and not the model.

Environment

  • Ruby version: 3.4.8
  • RubyLLM version: main (b229895)
  • Provider (OpenAI, Anthropic, etc.): OpenAI (Responses API), also reproduced on Azure OpenAI v1
  • OS: Linux

Proposed fix

Send strict: false explicitly in Responses::Tools.tool_for to preserve the Chat Completions behavior, and let tools opt in with provider_options strict: true (which already merges over the definition). I have a patch ready with specs and docs updates (upgrading guide note + a strict-mode section in tool-parameters) — happy to open the PR if you agree with defaulting to non-strict. If you'd rather keep strict-by-default, this at least deserves a call-out in the upgrading guide, since properly strict schemas also require every property in required and nullable types for optionals, which most existing tools won't have.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions