Skip to content

fix(agent): harden tool-call path against malformed relay responses - #4569

Merged
Re-bin merged 2 commits into
HKUDS:mainfrom
m11y:fix/tool-call-robustness
Jun 28, 2026
Merged

Re-bin merged 2 commits into
HKUDS:mainfrom
m11y:fix/tool-call-robustness

Conversation

@m11y

@m11y m11y commented Jun 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Some upstream relays can emit malformed tool-use responses where the tool name is missing or not a usable string. If such a call reaches execution or gets persisted into history, later turns can crash, fail provider validation, or keep replaying the same broken tool-call pattern. Compaction placeholder assistant messages can also nudge the model back toward failed tool-call demonstrations.

Changes

Hardens the tool-call path in two focused places:

  • Adds ToolCallRequest.has_valid_name() and drops response tool calls whose name is missing or empty before execution/persistence.
  • Retries once with a corrective note when every returned tool call was malformed, then falls back to a no-tools request if the retry is still malformed.
  • Skips malformed-name calls in tool hint formatting instead of letting hint generation crash the turn.
  • Cleans polluted model-facing history in ContextGovernor by removing malformed persisted tool calls and placeholder assistant messages before request shaping. Persisted transcript data is left untouched.

Provider parsers already synthesize missing tool IDs where needed, and argument shape validation remains in ToolRegistry, so this PR keeps the guard intentionally narrow around the malformed-name failure mode.

Testing

  • python -m pytest tests/agent/test_runner_governance.py tests/agent/test_tool_hint.py -q
  • python -m ruff check nanobot/agent/context_governance.py nanobot/agent/runner.py nanobot/providers/base.py nanobot/utils/tool_hints.py tests/agent/test_runner_governance.py tests/agent/test_tool_hint.py

Combine malformed tool-call handling with placeholder filtering and a
no-tools fallback so a relay that returns tool_use blocks with null
id/name/input can no longer crash a turn or permanently wedge a session.

Adapted to the ContextGovernor architecture (context governance now lives
in nanobot/agent/context_governance.py, not runner.py):

- ToolCallRequest.has_valid_name(): single source of truth for "usable
  name" (non-empty string).
- tool_hints.format_tool_hints(): skip tool calls with a non-string/empty
  name instead of raising AttributeError on the whole turn.
- ContextGovernor.strip_placeholder_assistant_messages() and
  strip_malformed_tool_calls() (plus the _tool_call_name_is_valid helper):
  history-cleaning staticmethods invoked at the START of
  prepare_for_model() — strip_placeholder, then strip_malformed, then the
  existing drop_orphan/backfill chain. Both only repair the model-facing
  copy and leave persisted history untouched (return a copy, or the same
  list when nothing changes). Also wired into runner's minimal-repair path.
- AgentRunner._drop_malformed_tool_calls(): returns
  (dropped, all_dropped, original_finish_reason); clears finish_reason to
  "stop" when all calls are dropped.
- AgentRunner._malformed_tool_call_retry_messages() + _request_model
  malformed_retry flag: when an all-dropped tool_calls response comes back,
  retry once with a corrective note; if the retry STILL comes back
  all-dropped, fall back to _request_no_tools for graceful text degradation.

Tests for the history-cleaning methods live with ContextGovernor in
tests/agent/test_runner_governance.py; response-layer and tool-hint tests
stay on AgentRunner / tool_hints.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@Re-bin Re-bin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This is a focused defensive fix for malformed tool-call names: bad calls are filtered before execution/persistence, polluted model-facing history is cleaned, and all-malformed responses get one corrective retry before falling back without tools.

I pushed a small follow-up to fix ruff formatting and align the retry note / PR body with the intentionally narrow guard. Local targeted validation passed:

  • python -m pytest tests/agent/test_runner_governance.py tests/agent/test_tool_hint.py -q
  • python -m ruff check nanobot/agent/context_governance.py nanobot/agent/runner.py nanobot/providers/base.py nanobot/utils/tool_hints.py tests/agent/test_runner_governance.py tests/agent/test_tool_hint.py

@Re-bin
Re-bin merged commit 7dc45ff into HKUDS:main Jun 28, 2026
@m11y
m11y deleted the fix/tool-call-robustness branch June 29, 2026 02:23
m11y added a commit to m11y/nanobot that referenced this pull request Jul 28, 2026
Temporary diagnostic to locate the intermittent upstream relay bug where a
tool_use block comes back with a missing/non-string name (finish_reason still
'tool_calls'), forcing HKUDS#4569's drop->retry->no-tools fallback and surfacing
degenerate replies like "call".

On both the streaming and non-streaming Anthropic paths, when any returned
tool_use has an empty name, dump the full request (system + messages + tool
names), the full response (per-block raw dump, so we can see whether id/input
survive), and the streamed content_block_start events (to tell whether the
relay drops the name in the SSE stream itself or only in the assembled
message) to ~/.nanobot/logs/malformed_toolcall_*.json.

On by default; opt out with NANOBOT_NO_MALFORMED_CAPTURE=1. Never raises.
To be replaced by the real recovery fix once the captured data pins down what
survives.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants