Skip to content

fix(judge): preserve tool argument structure when matching - #304

Merged
zpzjzj merged 1 commit into
alibaba:mainfrom
mavericksea-ai:fix/structured-tool-argument-matching
Oct 9, 2026
Merged

zpzjzj merged 1 commit into
alibaba:mainfrom
mavericksea-ai:fix/structured-tool-argument-matching

Conversation

@mavericksea-ai

Copy link
Copy Markdown
Contributor

Summary

Tool-argument rules currently compare formatted text, so paths: ["sales 2025.csv"] matches paths: ["sales", "2025.csv"]. A wrong file request passes the rule and can make CI succeed. In an offline benchmark, this reports 100% versus 100% and zero lift; structural comparison correctly reports 0% versus 100%, −1.00 lift, and exit code 1.

Related issues

Closes #303.

Changes

  • Compare each declared argument using JSON value encoding, preserving array boundaries, object members and scalar types.
  • Preserve matching across ordinary YAML integer and JSON float representations, and allow extra top-level actual arguments as before.
  • Cover both tool_called and tool_called_in_turn, plus structured collisions and compatibility cases; update the changelog.

Test plan

  • make test passes (race-enabled Go tests and 22 plugin tests)
  • make verify passes (fmt + vet + lint), using Go 1.25.13 with the repository's pinned tools
  • make fmt run
  • Offline real-CLI reproduction on main and published v0.12.0; eight regression subcases fail before and pass after

Notes for reviewers

The current Go 1.27 compiler produces export data unsupported by the pinned lint tool; make verify passes with the supported Go 1.25.13 toolchain. No module files or lint rules were changed. E2E-tag tests are not required for this change because no runner or e2e files changed; the separate reproduction runs the CLI end to end.

@CLAassistant

CLAassistant commented Oct 8, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@zpzjzj zpzjzj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zpzjzj
zpzjzj added this pull request to the merge queue Oct 9, 2026
Merged via the queue into alibaba:main with commit 0579a86 Oct 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rule-based tool argument matching conflates different arrays and can falsely pass CI

3 participants