Add tool call support to Mistral structured handler#965
Open
CamilleScholtz wants to merge 1 commit intoprism-php:mainfrom
Open
Add tool call support to Mistral structured handler#965CamilleScholtz wants to merge 1 commit intoprism-php:mainfrom
CamilleScholtz wants to merge 1 commit intoprism-php:mainfrom
Conversation
Mistral does not allow `response_format` and `tools` in the same request. This adds a two-phase approach to the structured handler: Phase 1: When tools are present, send the request with tools but without response_format. Handle tool calls in a loop until the model stops calling tools or max steps is exhausted. Phase 2: Send a final request without tools but with json_schema response_format to get the structured JSON output. Also upgrades from json_object mode (relies on system prompt instruction) to json_schema mode (strict server-side schema enforcement by Mistral). The implementation follows the same patterns as the Mistral text handler: reusable addStep() method, ExtractsText/ExtractsThinking traits, rate limit processing, and shouldContinue() for max steps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Mistral structured handler currently has no tool call support and uses the legacy
json_objectresponse format with a system prompt instruction for schema enforcement. This PR adds full tool calling with a two-phase approach and upgrades tojson_schemamode.The problem
Mistral does not allow
response_formatandtoolsin the same API request. The current structured handler has no tool support at all, it simply appends a system message with the JSON schema and usesjson_objectmode.The solution
A two-phase approach:
Phase 1 (tool calling): When tools are present, send requests with
tools/tool_choicebut withoutresponse_format. Handle tool calls in a loop until the model stops or max steps is exhausted.Phase 2 (structured output): Send a final request without tools but with
json_schemaresponse format for strict server-side schema enforcement.Edge cases handled:
json_schemaresponse format directlyresponse_formatwas omitted), so it's discarded and a follow-up request is sent without tools to get proper structured outputChanges
CallsTools,ExtractsText,ExtractsThinkingtraits (matching the Text handler)appendMessageForJsonMode()(system prompt hack) withjson_schemaresponse format (strict server-side enforcement)sendAndRespond()orchestration withhandleToolCalls()/handleStop()/sendStructuredResponse()addStep()for reuse across tool call and stop flows (matching Text handler pattern)rateLimitsin Meta viaprocessRateLimits()(was missing)additionalContentfromextractThinking()(was missing)How it follows the existing patterns
This implementation mirrors the Mistral Text handler's structure:
CallsTools,ExtractsText,ExtractsThinking,MapsFinishReason,ProcessRateLimits,ValidatesResponse)addStep()reusable method patternshouldContinue()logic (includingmaxSteps === 0as infinite)mapToolCalls()implementationAssistantMessage→ToolResultMessage→resetToolChoice)