fix: prevent unavailable tool names from leaking into model schemas#2072
Merged
fix: prevent unavailable tool names from leaking into model schemas#2072
Conversation
added 2 commits
March 19, 2026 10:04
When web_search/web_extract fail check_fn (no API key configured), their names were still leaking into tool descriptions via two paths: 1. execute_code schema: sandbox_enabled was computed from tools_to_include (pre-filter) instead of the actual available tools (post-filter), so the execute_code description listed web_search/web_extract as available sandbox imports even when they weren't. 2. browser_navigate schema: hardcoded description said 'prefer web_search or web_extract' regardless of whether those tools existed. The model saw these references, assumed the tools existed, and tried calling them directly — triggering 'Unknown tool' errors. Fix: compute available_tool_names from the filtered result set and use that for both execute_code sandbox listing and browser_navigate description patching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When
web_search/web_extractare unavailable (no API key configured), the model was still seeing references to them in two places:1.
execute_codeschema —sandbox_enabledwas computed fromtools_to_include(the pre-filter toolset), so the execute_code description listedweb_search(query, limit)as an available sandbox import even when the check_fn had filtered it out.2.
browser_navigateschema — Static description said "prefer web_search or web_extract (faster, cheaper)" regardless of tool availability.The model saw these references, assumed the tools existed, and called them directly — triggering repeated
Unknown tool 'web_search' — sending error to model for self-correctionerrors (reported by user kidtaroi).Fix
Compute
available_tool_namesfrom the post-filterfiltered_toolsand use that set for:tools_to_include)Test plan
get_tool_definitions()inmodel_tools.py— single file change