Repository navigation
fix(executor): coerce runtime lists to predicates at WHERE/FILTER positions - #76
Conversation
|
Security review (security-review skill) Reviewed the executor predicate-policy change: the new No issues found. Security review: ✅ no issues found |
| let mut filtered = Vec::with_capacity(rows.len()); | ||
| for row in rows { | ||
| if eval(graph, &row, predicate, params, cypher_version)?.truthy() == Some(true) { | ||
| if crate::eval::predicate_value_holds(&eval( |
There was a problem hiding this comment.
[Medium] Error handling: The new predicate policy is not the single path for every MATCH/WITH WHERE: the aggregating-WITH HAVING arm (relate/relate-executor/src/exec/aggregate/group.rs:262) and the shortest-path route filter (relate/relate-executor/src/exec/gpm/shortest_route/route.rs:431) still read Value::truthy() and silently drop the row/group.
Why this matters: the same clause now converts a runtime list predicate (or raises InvalidArgumentType for another non-boolean) on one plan but silently yields no match on another — e.g. MATCH p = shortestPath((a)-[r*]->(b)) WHERE r routes through route.rs and matches nothing — so results depend on the physical plan, and the HAVING arm additionally swallows every agg_eval error.
Suggested fix: call predicate_value_holds and propagate its Err in both places (route.rs can use predicate_holds directly; HAVING: if !predicate_value_holds(&agg_eval(...)?)? { continue; }).
| } | ||
| } | ||
|
|
||
| /// Whether an evaluated boolean-context predicate holds: the same runtime |
There was a problem hiding this comment.
[Low] Idiomatic Rust: The bool_operand doc directly above still says the list/path conversion is "scoped to exactly these four operators" and that "Every other requiring site reads bool_or_null directly" — the new predicate_value_holds below is such a site, and the updated ADR-0458 §7 names the predicate positions.
Why this matters: two adjacent policy comments now contradict each other, so a later change can re-derive the rule wrongly (for example drop the List arm from bool_operand or route a predicate position through bool_or_null, silently bringing back the no-match behavior).
Suggested fix: update the bool_operand (and bool_or_null) docs to name the predicate positions as the second consumer of the conversion.
|
Rust review (rust-reviewer skill, full review) Full review of the current diff — the No issues found. Rust review: ✅ no issues found |
| assert!( | ||
| err.to_string().contains("requires a boolean"), | ||
| "a non-boolean predicate is the typed type error, not a silent no-match: {err}" | ||
| ); |
There was a problem hiding this comment.
[Medium] boundaries (std-5): The test is named and documented for the typed TypeError class (ADR-0511), but it only scans the rendered message. The public typed classification (relate::binding_error_metadata / `relate::error_
Suggested fix: Assert the class through the existing typed path instead of the bespoke substring — assert_eq!(error_class_literal(&mut c, "MATCH (n:CoerceNumber) WHERE n.num RETURN n"), "TypeError") (or fold this dynamic-property row into type_error_classes.rs's boolean-site table) — keeping the message-fragme
finding F-1-1
|
Stellar review (stellar-review skill) Round 9 · head Still open from earlier rounds
Stellar review: |
df16fd7 to
ce35872
Compare
|
Round 1 fix round — head
Gates on the re-squashed tree: fmt, workspace clippy, Link check: outage-class, not a finding — the job died building its own image ( |
| ); | ||
| if !matches!(keeps, Ok(value) if value.truthy() == Some(true)) { | ||
| )?; | ||
| if !crate::eval::predicate_value_holds(&keeps)? { |
There was a problem hiding this comment.
[Medium] boundaries (std-5): The new ? makes a HAVING predicate-evaluation error abort the query, but the field contract this implementation is documented by (GroupSubclauses::having, same file: 'A predicate error drops the g
Suggested fix: Update the GroupSubclauses::having contract to the new propagation rule (false/null drops the group; a non-boolean is the typed error) — or restore drop-on-error if that was intended — so the field's documented invariant matches aggregate_ex.
finding F-3-1
| assert_eq!( | ||
| error_class_literal( | ||
| &mut c, | ||
| "MATCH (n:CoerceAgg) WITH n.num AS num, count(*) AS k WHERE num RETURN num", |
There was a problem hiding this comment.
[Medium] boundaries (std-5): Both WHERE predicates (ls, num) carry no aggregate call, so project.rs's split (having = where_clause.filter(|w| has_agg && is_agg_expr(w))) leaves having = None and routes them to the plain
Suggested fix: Make the HAVING predicate itself aggregate-bearing and still list/non-boolean-valued (fold the group's value through an aggregate call so having is Some), or unit-test aggregate_ex with having = Some(...); if the intent is the post-merge filter, rename the test and drop the HAVING claims.
finding F-3-2
ce35872 to
9c51fe5
Compare
| /// list/path conversion the boolean operators apply (an empty list is false, | ||
| /// a non-empty list and any path true), a null never holds, and any other | ||
| /// non-boolean is the typed `InvalidArgumentType` a statically known one is | ||
| /// rejected with. The MATCH/WHERE, WITH/FILTER, HAVING and GPM route-filter |
There was a problem hiding this comment.
[Medium] Error Handling: The new predicate helper is not yet the "one policy" this doc claims: several predicate positions still read Value::truthy(), which silently turns every non-boolean into a no-match instead of converting or raising.
Why this matters: exec/gpm/qpp.rs:288,294 (QPP element/path WHERE), exec/procedures.rs:176 (CALL … YIELD … WHERE), exec/catalog.rs:268 (SHOW … WHERE), and eval/mod.rs:545,655 (comprehension WHERE) keep the old silent-false behavior, so e.g. MATCH (a)((n) WHERE n.coll){1,2}(b) drops rows while the identical inline predicate on a plain MATCH now converts, and a numeric procedure yield filters to zero rows instead of the typed InvalidArgumentType.
Suggested fix: route those sites through predicate_value_holds (or bool_or_null where the position is intentionally strict) and update this doc plus ADR-0458 §7; if they are deliberate exceptions, name them there so the "one policy" claim holds.
| }, | ||
| ) | ||
| .unwrap_err(); | ||
| assert!(err.to_string().contains("boolean"), "{err}"); |
There was a problem hiding this comment.
[Medium] comparison (std-5): The new HAVING test discriminates the rejection by a message substring, while the sibling public test in this change pins the class through error_class_literal (F-1-1's accepted typed-path remedy) a
Suggested fix: Assert the class through the available typed path — matches!(err, Error::InvalidArgumentType(_)) and/or relate_error::error_info(&err).neo4j_code == relate_error::TYPE_ERROR_CODE, as eval/tests.rs does — keeping the message check only alongside the typed one.
finding F-4-1
9c51fe5 to
d513df8
Compare
| /// convert-or-raise policy the ordinary WHERE positions read | ||
| /// (ADR-0458 §7, [`crate::eval::predicate_value_holds`]). `None` for | ||
| /// every other shape. | ||
| pub having: Option<&'a Expr>, |
There was a problem hiding this comment.
[Medium] boundaries (std-5): This is the canonical HAVING error contract now: aggregate_ex propagates the predicate fold error and the new test pins the abort, but the lenient-site contract in aggregate/stream.rs (`SiteAcc::F
Suggested fix: State the HAVING outcome once: make the lenient-site comment defer to this contract (it should only say the parked error surfaces when the site is read), or update it to 'a HAVING read aborts the statement'. Do not restate the outcome at both sites.
finding F-5-1
d513df8 to
84fd199
Compare
| // count(*) is an INTEGER per group: the typed rejection. | ||
| let err = aggregate_ex( | ||
| &graph, | ||
| group_with_vals(&[Value::Int(1)]), |
There was a problem hiding this comment.
High, Compilation: aggregate_ex takes RowFeed<'_>, not Vec<Row> — this new test does not compile.
Why this matters: group_with_vals returns Vec<Row>, and RowFeed is the enum Held(Vec<Row>) | Unwind(..) (exec/unwind.rs), so both new aggregate_ex calls (line 651 and line 676) pass the wrong type and fail cargo test / cargo clippy --all-targets for the crate.
Suggested fix: import super::unwind::RowFeed and wrap the feed: RowFeed::Held(group_with_vals(&[Value::Int(1)])), RowFeed::Held(rows), and RowFeed::Held(vec![]) for the empty case.
| /// the site is read, where the key orders as null and HAVING drops the | ||
| /// group — the documented policy — instead of failing the statement. | ||
| /// the site is read. The read's outcome is stated once at the | ||
| /// canonical contract ([`GroupSubclauses::having`], ADR-0458 §7); an |
There was a problem hiding this comment.
Low, Error Handling (stale doc): The sibling Slot::lenient doc (lines 79-80 of this file) still says "a fold error there is not a statement failure", but this PR makes a HAVING fold error abort the statement.
Why this matters: Slot::lenient now contradicts the updated SiteAcc::Failed doc right above it, so the two comments disagree on the error policy a future change would follow.
Suggested fix: update lenient's doc to say the error is parked so an ORDER BY key can order as null, while HAVING surfaces it through predicate_value_holds.
84fd199 to
9f0d7f1
Compare
| having: Some(&collect), | ||
| import_scope: None, | ||
| }, | ||
| ) |
There was a problem hiding this comment.
[Medium] branching (std-3): This makes the test a third way to drive the one HAVING path: the file already has try_exec_cypher for the query-level contract (the neighbouring `a_having_only_aggregate_that_fails_to_fold_aborts_t
Suggested fix: Express the three cases through the existing try_exec_cypher harness — UNWIND [1] AS x WITH count(x) AS c WHERE count(x) RETURN c for the typed rejection, the same with WHERE collect(x) over [1, 2] and over [] for the kept/dropped groups — which reaches the identical `GroupSubclauses::havi
finding F-7-1
| // (live: `ANY SHORTEST ... -[r WHERE r.ls]-> ...` accepts a | ||
| // non-empty list), a non-boolean is the typed rejection live | ||
| // raises (22N01). | ||
| if !crate::eval::predicate_value_holds(&crate::eval::eval_match_predicate( |
There was a problem hiding this comment.
[Low] Test coverage: The new list/non-boolean branch in Route::filter_ok has no test — shortest_route/route_tests.rs and reject_tests.rs cover length and NONE(...) predicates only, so the live case cited in the comment (ANY SHORTEST ... -[r WHERE r.ls]-> ...) is unpinned.
Why this matters: AGENTS.md §4 requires a test for every behavior change; a regression in this route-only predicate path would ship unnoticed.
Suggested fix: add a route test (using the existing on_route vs GPM parity helpers) with a non-empty-list relationship property and a non-boolean case.
| /// predicate positions read this one helper; the list-quantifier's inner | ||
| /// WHERE keeps [`bool_or_null`]. | ||
| /// Named apart from [`predicate_holds`] (the expression-level form). | ||
| pub(crate) fn predicate_value_holds(v: &Value) -> Result<bool> { |
There was a problem hiding this comment.
[Medium] Error handling: The helper's "one predicate policy" claim does not hold yet: gpm_qpp_iteration_predicates_ok (exec/gpm/qpp.rs, both the element and group WHERE), RunGuards::predicate_admits (exec/kernels/frontier.rs, the NFA group WHERE), the comprehension WHEREs in eval/mod.rs (retain_projected and the map-comprehension path), and SHOW ... WHERE (exec/catalog.rs) still hand-spell .truthy() == Some(true).
Why this matters: a runtime list predicate at those positions silently prunes the row instead of converting (and any other non-boolean silently filters instead of raising the typed InvalidArgumentType), so the same predicate returns different rows depending on the clause or engine path (auto QPP vs the forced NFA rung) — the exact silent-false behavior this PR removes from WHERE/FILTER.
Suggested fix: evaluate those sites through crate::eval::predicate_value_holds (or predicate_holds where the site has an Expr), and add them to the helper's doc/ADR list so the canonical policy is exhaustive.
… positions A property read has no statically known type, so Neo4j's front-end listCoercedToBooleanCheck cannot fire and the runtime converts the value in a boolean predicate position. Relate filtered such a row out instead (MATCH (n) WHERE n.coll AND n.bool dropped every node, because the planner splits the conjunction and each half was read through Value::truthy()). The MATCH/inline-WHERE path (eval::predicates::predicate_holds) and the WITH/FILTER filter (exec::project::filter_where) now read one helper, eval::predicate_value_holds, which applies the same list/path conversion the boolean operators use: an empty list is false, a non-empty list and any path true, null never holds, and any other non-boolean is the typed InvalidArgumentType a statically known one is already rejected with. The list-quantifier's inner WHERE keeps the strict boolean-or-null rule. ADR-0458 §7 records the predicate-position scope. Live-pinned on Neo4j 2026.10.1: MATCH (n:LABEL) WHERE n.coll AND n.bool returns the non-empty/true node, and FILTER n.coll passes every non-empty list. Co-Authored-By: Claude Code <noreply@anthropic.com> Review round: HAVING and the GPM route filter read the one predicate policy (live 22N01 evidence for the non-boolean raise, ANY SHORTEST leg-WHERE list conversion); bool_operand doc covers the predicate positions; the TypeError test asserts the typed class via error_class_literal. Round 3: the having field contract documents the convert-or-raise rule; a focused aggregate_ex unit test exercises the HAVING arm directly (collect conversion, count rejection, phantom-group drop); the integration test asserts the returned group value via row(). Round 7: the QPP leg and whole-pattern WHEREs (gpm/qpp.rs), the pattern engines per-trial guard (kernels/frontier.rs), and the list/map comprehension WHEREs read the one predicate policy (live-probed: QPP and comprehension positions convert a runtime list and raise 22N01 on a non-boolean; SHOW...WHERE stays a recorded divergence — live statically rejects against column types); ADR-0458 section 7 scope updated; the HAVING test expressed through the try_exec_cypher query-level harness; route/QPP/comprehension branches covered by a live-pinned integration test.
9f0d7f1 to
b13f075
Compare
| if !predicate_value_holds(&eval(graph, &r, pred, params, cypher_version)?)? { | ||
| continue; | ||
| } | ||
| } |
There was a problem hiding this comment.
[Medium] branching (std-3): The changed line now reads the one predicate policy, but it leaves the optional-WHERE gate hand-spelled right next to eval/predicates.rs::where_clause_holds, whose doc declares itself the one spelli
Suggested fix: Give the comprehension positions the one optional-WHERE gate: call where_clause_holds (widening it to a value-level form beside predicate_value_holds if the comprehension path must keep plain eval) here and in eval_map_comp, collapsing each loop body to a single `if !gate(...)? { continue; }
finding F-8-1
| if !predicate_value_holds(&eval(graph, &r, pred, params, cypher_version)?)? { | ||
| continue; | ||
| } | ||
| } |
There was a problem hiding this comment.
[Medium] branching (std-3): This is the same hand-spelled optional-WHERE gate as retain_projected (548-552) and as the canonical where_clause_holds gate: the diff routes the defaulted predicate value through the one policy h
Suggested fix: Share the gate with retain_projected: route both loops through one helper (the existing where_clause_holds, or its value-level sibling if plain eval must stay) so the absent/imposed rule has one spelling and each loop reads if !gate(...)? { continue; }.
finding F-8-2
| )?)? { | ||
| return Ok(false); | ||
| } | ||
| } |
There was a problem hiding this comment.
[Medium] branching (std-0): The changed hunks adopt the predicate policy in two copies of the same gate — the element-WHERE loop at 287-297 and the qpp.where_clause block at 299-309 — and the element-WHERE vocabulary (Node/`
Suggested fix: Add a single qpp_predicates(qpp: &QuantifiedPathPattern) -> impl Iterator<Item = &Expr> that chains each element's where_clause with qpp.where_clause, and use it in both gpm_qpp_iteration_predicates_ok (one loop, one gate, Ok(true) at the end) and qpp_unbound_predicate_names (vars collec
finding F-8-3
|
Loop-bounded at this round. The round's findings are all one class — a second/third spelling of a policy this PR itself introduced (structure/branching consolidation), no correctness or security content; the code is live-probed and the set-equality gates are exact. Per the repo's loop-bound rule, these are batched into LABS-1270 (review-tasks, Agent Backlog, with a Files: section) and the PR lands. |
|
Loadtest baselines comparison
Loadtest baseline comparisonFrozen baselines vs current reports; bands match the Status: warning Pairs: fraud-loadtest: report-missing · identity-loadtest: report-missing fraud-loadtest
identity-loadtest
|
|
Memory controls and workspace memory calibration
(comparison document unavailable) |
|
Memory shape
(comparison document unavailable) |
Oplane Security ReviewOplane identified security requirements relevant to this PR and checked the implementation against them. All requirements are satisfied. Review the generated threat model.
Change comment settings · Set up Oplane in your agent · Workflow guidance |
Fixes one equality/misc corpus row and triages the family's remainder (live-probed against Neo4j 2026.10.1).
Fixed —
correctness/ComparisonsAndNull.feature:45: predicate positions (MATCH/inlineWHEREandWITH/FILTER) readValue::truthy(), which isNonefor a list — the planner splitn.coll AND n.boolso the list property filtered to nothing. Neo4j has no static type for a property read and converts at runtime.eval::predicate_value_holds(&Value) -> Result<bool>— the boolean-operator list/path conversion (empty list false, non-empty list and any path true), null never holds, other non-booleans are the typedInvalidArgumentTypepredicate_holds(MATCH/inline WHERE) andfilter_where(WITH/FILTER) both read itpredicate_list_coercion.rs: the scenario rows, FILTER, non-boolean TypeError, null predicate)Known limit (pre-existing, stated for the record): Neo4j's static checker rejects a statically-known list predicate (
WITH [1,2] AS xs WHERE xs→ 22N37); Relate has no variable type inference, so such a predicate converts (non-empty → row). No corpus scenario covers it; no inference change in this PR.Rows triaged out (no engine change, evidence in the review):
TernaryEqualityAcceptance.feature:103— invalid expectation (harness cell parser makes the string key"null"; Neo4j and Relate both returneq=false)Expression.feature:201— feature table lists the distinct row; both engines return 3 rowsCIP-146/use-braces.feature:33anddb.propertyKeyspersistence (BuiltInProcedure.feature:348/:369) — design decisions, ticketed separately (second queryable namespace / monotone property-key token store needing a checkpoint-format ADR)CIP-162/labels-function-alternative.feature:357— the CIP's own doc contradicts its normative overwrite rule; the feature never shipped in Neo4j (parse-time 42I06), no live oracleGates: fmt, workspace clippy, executor suite (1363), relate + relate-cli suites green.