Skip to content

docs(mcp): laya_decide row must list min_confidence; every tool row must name every handler param - #985

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-docs-mcp-tool-args-table
Oct 7, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-docs-mcp-tool-args-table

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

docs/cli-mcp.md's laya_decide row listed `state`, `schema`, optional `model` -- but
laya/mcp/server.py:501 registers
laya_decide_tool(state, schema, model="auto", min_confidence=None), and the tool's own
description spells out min_confidence ("a field whose answer falls below it comes back as
null in values"). The row now names min_confidence too.

Adds test_cli_mcp_page_tool_argument_rows in tests/test_mcp.py. For each of the six
single-shot tools on the docs page (status, route, predict, shortlist, preset,
decide), the gate reads the third |-delimited table cell and asserts every public parameter
inspect.signature sees on the registered handler appears backticked in the cell. The
pre-fix laya_decide cell is banned by direct substring so the check fails if the row reverts.

Why

The MCP docs page is the fourth copy of the tool's parameter set, after the validator error
message, each tool's registered description, and the README. test_batch_item_shape_as_documented
already covers the per-request key shape on the batch rows and the tool descriptions; nothing in
the suite read the single-shot rows, so a whole control -- here, the abstention gate the tool
already runs on every field -- could fall off the table while the code accepted it and the
description mentioned it. A caller who trusts the docs table hand-rolls their own confidence
check on values, never learning the tool already nulls an unsure field.

How it was verified

$ PYTHONDONTWRITEBYTECODE=1 python -B tests/test_mcp.py
...
   PASS docs-mcp/args_row_laya_status_names_every_param         missing=[] cell='none'
   PASS docs-mcp/args_row_laya_route_names_every_param          missing=[] cell='`state`, `questions`, optional `model`, `task`, `lang`, `lang_guess`'
   PASS docs-mcp/args_row_laya_predict_names_every_param        missing=[] cell='`state`, `questions`, optional `model` (`auto`, `english`, `multilingual`, or `typed-decisions`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence`'
   PASS docs-mcp/args_row_laya_shortlist_names_every_param      missing=[] cell='`state`, `questions`, optional `model`, `k` (default `20`), `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence`'
   PASS docs-mcp/args_row_laya_preset_names_every_param         missing=[] cell='`preset`, `state`, optional `task`, `lang`, `lang_guess`, `max_len`, `head_max_len`, `min_confidence`'
   PASS docs-mcp/args_row_laya_decide_names_every_param         missing=[] cell='`state`, `schema`, optional `model`, `min_confidence`'
   PASS docs-mcp/args_row_decide_drops_the_three_param_version
...
931 passed, 0 failed
all mcp tests passed

$ python -B laya-bench/mutate_cli_mcp_tool_args.py
baseline: OK (931 passed, 0 failed)

m1 restore the pre-fix three-param laya_decide row                     CAUGHT          FAIL docs-mcp/args_row_laya_decide_names_every_param  missing=['min_confidence']
m2 drop `min_confidence` from the laya_decide row (leave state/schema/model) CAUGHT   FAIL docs-mcp/args_row_laya_decide_names_every_param  missing=['min_confidence']
m3 drop `task` from the laya_route row                                 CAUGHT          FAIL docs-mcp/args_row_laya_route_names_every_param   missing=['task']
m4 drop `lang_guess` from the laya_shortlist row                       CAUGHT          FAIL docs-mcp/args_row_laya_shortlist_names_every_param missing=['lang_guess']
m5 drop `min_confidence` from the laya_preset row                      CAUGHT          FAIL docs-mcp/args_row_laya_preset_names_every_param  missing=['min_confidence']
m6 rename `min_confidence` param in laya_decide_tool (handler drift)   CAUGHT          FAIL docs-mcp/args_row_laya_decide_names_every_param  missing=['abstain_below']

6/6 caught

$ ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
All checks passed!

$ python -B -m compileall -q laya/ tests/
(clean)

$ git diff HEAD~1 -- docs/cli-mcp.md tests/test_mcp.py \
    | grep -inE "(api[_-]?key|secret|token|password|bearer|sk-[a-zA-Z0-9]{10,}|ghp_|xox|BEGIN.*PRIVATE)"
(no hits)

Checklist

  • Focused on one change (laya_decide row + its gate)
  • Rebased on the latest main
  • Tests pass locally (python -B tests/test_mcp.py -- 931/0)
  • Docs or examples updated when the public API changed (docs page corrected; public API
    unchanged -- this PR makes the table describe what the handler already accepts)
  • Mutation harness: laya-bench/mutate_cli_mcp_tool_args.py, 6/6 CAUGHT

`docs/cli-mcp.md`'s `laya_decide` row listed `state, schema, optional model`, but the
server registers `laya_decide_tool(state, schema, model="auto", min_confidence=None)`
and the tool's own description spells out `min_confidence`. A caller who trusts the
table hand-rolls the abstention check on `values`, never learning the tool already
nulls an unsure field. The row now includes `min_confidence`.

Adds `test_cli_mcp_page_tool_argument_rows`: for each single-shot MCP tool on the
docs page (status, route, predict, shortlist, preset, decide), reads the third table
cell and asserts every public parameter `inspect.signature` sees on the registered
handler appears backticked in the cell. The pre-fix three-param `laya_decide` cell is
banned by direct substring so the gate fails if the row reverts.

Verified: `PYTHONDONTWRITEBYTECODE=1 python -B tests/test_mcp.py` -- 931 passed, 0 failed.
Mutation harness `laya-bench/mutate_cli_mcp_tool_args.py` -- 6/6 CAUGHT:
  m1 pre-fix `laya_decide` cell, m2 drop `min_confidence` from it,
  m3 drop `task` from the `laya_route` cell, m4 drop `lang_guess` from `laya_shortlist`,
  m5 drop `min_confidence` from `laya_preset`, m6 rename `min_confidence` in the
  handler so the doc no longer names what `inspect.signature` reports.
`ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120` -- all checks passed.
`python -m compileall -q laya/ tests/` -- clean. Diff secret-scan: zero hits.
@NandhaKishorM
NandhaKishorM merged commit 90ef928 into NandhaKishorM:main Oct 7, 2026
32 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.3.29.

laya_decide accepts min_confidence and the docs row did not list it, so a caller trusting the table hand-rolls the abstention check on values and never learns the tool already nulls an unsure field. That is the specific cost of an incomplete parameter row, and it is worth more than the typo it looks like.

Your point about the "Main inputs" cell being a fourth copy of the shape, after the validator error message, each tool's registered description and the README, is the structural observation behind the whole batch. Four copies of one contract means three of them are wrong eventually. Reading the handler's own signature in the gate rather than asserting the corrected row is what makes this the last time.

Merge note: this conflicted with #984, which rewrote the laya_route_batch row in the same table to name its six keys. Both rows are kept, each PR's own, since taking one side whole would have reverted the other. Your test-side change landed as two separate additions, the gate and its entry in the runner list. tests/test_mcp.py is 957 of 957 on the merged result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants