Skip to content

Add tool call support to Mistral structured handler#965

Open
CamilleScholtz wants to merge 1 commit intoprism-php:mainfrom
CamilleScholtz:feat/mistral-structured-tool-calls
Open

Add tool call support to Mistral structured handler#965
CamilleScholtz wants to merge 1 commit intoprism-php:mainfrom
CamilleScholtz:feat/mistral-structured-tool-calls

Conversation

@CamilleScholtz
Copy link

@CamilleScholtz CamilleScholtz commented Mar 18, 2026

The Mistral structured handler currently has no tool call support and uses the legacy json_object response format with a system prompt instruction for schema enforcement. This PR adds full tool calling with a two-phase approach and upgrades to json_schema mode.

The problem

Mistral does not allow response_format and tools in the same API request. The current structured handler has no tool support at all, it simply appends a system message with the JSON schema and uses json_object mode.

The solution

A two-phase approach:

  1. Phase 1 (tool calling): When tools are present, send requests with tools/tool_choice but without response_format. Handle tool calls in a loop until the model stops or max steps is exhausted.

  2. Phase 2 (structured output): Send a final request without tools but with json_schema response format for strict server-side schema enforcement.

Edge cases handled:

  • No tools - sends a single request with json_schema response format directly
  • Tools present but model doesn't call them - the response has no schema enforcement (since response_format was omitted), so it's discarded and a follow-up request is sent without tools to get proper structured output
  • Max steps exhausted - forces a final structured response request

Changes

  • Added CallsTools, ExtractsText, ExtractsThinking traits (matching the Text handler)
  • Replaced appendMessageForJsonMode() (system prompt hack) with json_schema response format (strict server-side enforcement)
  • Added sendAndRespond() orchestration with handleToolCalls() / handleStop() / sendStructuredResponse()
  • Extracted addStep() for reuse across tool call and stop flows (matching Text handler pattern)
  • Included rateLimits in Meta via processRateLimits() (was missing)
  • Included additionalContent from extractThinking() (was missing)

How it follows the existing patterns

This implementation mirrors the Mistral Text handler's structure:

  • Same trait usage (CallsTools, ExtractsText, ExtractsThinking, MapsFinishReason, ProcessRateLimits, ValidatesResponse)
  • Same addStep() reusable method pattern
  • Same shouldContinue() logic (including maxSteps === 0 as infinite)
  • Same mapToolCalls() implementation
  • Same message sequencing (AssistantMessageToolResultMessageresetToolChoice)

Mistral does not allow `response_format` and `tools` in the same
request. This adds a two-phase approach to the structured handler:

Phase 1: When tools are present, send the request with tools but
without response_format. Handle tool calls in a loop until the model
stops calling tools or max steps is exhausted.

Phase 2: Send a final request without tools but with json_schema
response_format to get the structured JSON output.

Also upgrades from json_object mode (relies on system prompt
instruction) to json_schema mode (strict server-side schema
enforcement by Mistral).

The implementation follows the same patterns as the Mistral text
handler: reusable addStep() method, ExtractsText/ExtractsThinking
traits, rate limit processing, and shouldContinue() for max steps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant