Repository navigation
perf(executor): the shortest route probes its seek endpoints at the pair bound instead of listing every match (ADR-0555, LABS-1318) - #131
Conversation
|
Security review (security-review skill) The diff is an executor optimization in the shortest-route anchor resolution: endpoint probes now stop at No issues found. Security review: ✅ no issues found |
…air bound instead of listing every match (ADR-0555, LABS-1318) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
c9e210e to
d70720a
Compare
|
Stellar review (stellar-review skill) Round 3 · head Still open from earlier rounds
Stellar review: |
| /// A bound node, or a seeked list of at most [`SEEK_PAIR_LIMIT`]. | ||
| Listed(Side), | ||
| /// A seek matching more than [`SEEK_PAIR_LIMIT`] nodes; the list was | ||
| /// never built. |
There was a problem hiding this comment.
[Low] Documentation / allocation: Probe::Over claims "the list was never built", but the cap only reaches the residual scan.
Why this matters: Graph::lookup_nodes_upto_at (graph/index_lookup.rs:57-63) exempts index-served arms, so with a RANGE index on the endpoint's property the probe still materializes every match and then drops it; the (Probe::Over, Probe::Over) arm at route.rs:291 then re-seeks both sides in full, doing twice the work the pre-change code did once. The anchor comment at route.rs:272 ("the over side stays unbuilt") has the same over-claim.
Suggested fix: state the exemption in Probe::Over/lookup_nodes_upto_at ("an over-bound scan stops at cap+1; an index-served side answers in full"), or have Probe carry the list it already built instead of re-seeking it.
|
Rust review (rust-reviewer skill, full review) Reviewed the shortest-route endpoint capping in full: the new No issues found. Rust review: ✅ no issues found |
Oplane Security ReviewOplane reviewed this PR and did not identify any security requirements that need attention. Review the generated threat model. Change comment settings · Set up Oplane in your agent · Workflow guidance |
| }); | ||
| ids.sort(); | ||
| ids | ||
| let mut matched = Vec::new(); |
There was a problem hiding this comment.
[Medium] Performance: scan_match now collects matches into a second Vec on every call, including the uncapped path (cap == None) that all existing callers use.
Why this matters: this is the fallback the shared node-candidate funnel takes for every non-index-served label/property match (including per-row inner patterns like (n:Person {name: …})). The old code filtered the candidate list in place with retain; the new code keeps the original ids buffer alive while matched grows by push, so a non-selective scan holds two vectors at once (up to ~3x the transient buffer) and pays the growth reallocs on a path the rest of the engine reads per row.
Suggested fix: branch on cap — when cap.is_none() keep the old in-place ids.retain(...) + ids.sort(); use the early-exit loop only for Some(cap).
Spec-harness coverage — 🟡
Rerun |
|
Memory controls and workspace memory calibration
Memory controls and workspace memory calibrationStatus: improvement workload-indexed — processing_rss_min_bytesVerdict: improvement Given: a ledger at graph scale 10 loaded with the limits off, then reopened with --max-connections 0 and --concurrent-logical-workspace-limit 53.24 MB. When: 8 warmup operations run uncounted. Then 16 Bolt clients share 40 operations at 80:20, reads mix, indexes indexed. Then: every connection is accepted. A query that does not fit the workspace pool counts as limited. A write-write conflict counts as failed. Any other error fails the run. Outcome: reads succeeded 32, limited 0, failed 0; writes succeeded 8, limited 0, failed 0; connection refusals 0; workspace refusals 0. Measured 12.31 MB against the envelope 14.80 MB – 20.16 MB. Investigate whether this improvement is a false positive. When it is real, re-freeze the envelope with the create-loadtest-baselines workflow. sort — max_cgroup_to_logical_ratioVerdict: improvement Given: a fresh server per observation, admission on, a hold of 2500 ms, and 1 repeats. Families: sort, expand, aggregation, distinct, and create. When: each observation parks its sessions on RETURN 1, then holds one workspace family at the target reservation. The shapes are one holder at 1.00 MB, 4.00 MB, and 16.00 MB, then the same 16.00 MB across two and four holders. Then: each observation records VmRSS and cgroup deltas against the live held logical bytes. The child stays alive. Outcome: Expected cgroup ratio 0.999x–1.082x. A reading may sit 5% outside that range. Measured 1.24 MB, ratio 0.901x. Investigate whether this improvement is a false positive. When it is real, re-freeze the envelope with the create-loadtest-baselines workflow. expand — max_rss_to_logical_ratioVerdict: improvement Given: a fresh server per observation, admission on, a hold of 2500 ms, and 1 repeats. Families: sort, expand, aggregation, distinct, and create. When: each observation parks its sessions on RETURN 1, then holds one workspace family at the target reservation. The shapes are one holder at 1.00 MB, 4.00 MB, and 16.00 MB, then the same 16.00 MB across two and four holders. Then: each observation records VmRSS and cgroup deltas against the live held logical bytes. The child stays alive. Outcome: Expected RSS ratio 2.099x–2.147x. A reading may sit 5% outside that range. Measured 32.98 MB, ratio 0.970x. Investigate whether this improvement is a false positive. When it is real, re-freeze the envelope with the create-loadtest-baselines workflow. expand — max_cgroup_to_logical_ratioVerdict: improvement Given: a fresh server per observation, admission on, a hold of 2500 ms, and 1 repeats. Families: sort, expand, aggregation, distinct, and create. When: each observation parks its sessions on RETURN 1, then holds one workspace family at the target reservation. The shapes are one holder at 1.00 MB, 4.00 MB, and 16.00 MB, then the same 16.00 MB across two and four holders. Then: each observation records VmRSS and cgroup deltas against the live held logical bytes. The child stays alive. Outcome: Expected cgroup ratio 2.072x–2.215x. A reading may sit 5% outside that range. Measured 33.00 MB, ratio 0.970x. Investigate whether this improvement is a false positive. When it is real, re-freeze the envelope with the create-loadtest-baselines workflow. distinct — max_rss_to_logical_ratioVerdict: improvement Given: a fresh server per observation, admission on, a hold of 2500 ms, and 1 repeats. Families: sort, expand, aggregation, distinct, and create. When: each observation parks its sessions on RETURN 1, then holds one workspace family at the target reservation. The shapes are one holder at 1.00 MB, 4.00 MB, and 16.00 MB, then the same 16.00 MB across two and four holders. Then: each observation records VmRSS and cgroup deltas against the live held logical bytes. The child stays alive. Outcome: Expected RSS ratio 1.256x–1.322x. A reading may sit 5% outside that range. Measured 29.53 MB, ratio 1.073x. Investigate whether this improvement is a false positive. When it is real, re-freeze the envelope with the create-loadtest-baselines workflow. |
|
Memory shape
Memory shapeStatus: ok All memory shapes held. 🟢 baseGiven: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling. When: the base shape is loaded to 0 nodes and 0 relationships, then checkpointed. The child RSS is sampled. Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries. Baseline: logical 0.00 MB – 0.00 MB, disk 0.00 MB – 0.00 MB, attributed RSS 0.00 MB – 0.00 MB, ratio 0.000x–0.000x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes. Validated: logical 0.00 MB, disk 0.00 MB, attributed RSS 0.00 MB, ratio 0.000x. Report: logical, disk, attributed RSS, and the ratio held. flatGiven: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling. When: the flat shape is loaded to 120000 nodes and 0 relationships, then checkpointed. The child RSS is sampled. Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries. Baseline: logical 26.02 MB – 26.02 MB, disk 1.95 MB – 1.95 MB, attributed RSS 36.49 MB – 36.93 MB, ratio 1.402x–1.419x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes. Validated: logical 26.02 MB, disk 1.95 MB, attributed RSS 37.20 MB, ratio 1.430x. Report: logical, disk, attributed RSS, and the ratio held. fanoutGiven: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling. When: the fanout shape is loaded to 52016 nodes and 52000 relationships, then checkpointed. The child RSS is sampled. Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries. Baseline: logical 28.62 MB – 28.62 MB, disk 1.75 MB – 1.75 MB, attributed RSS 34.93 MB – 36.61 MB, ratio 1.220x–1.279x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes. Validated: logical 28.62 MB, disk 1.76 MB, attributed RSS 36.64 MB, ratio 1.280x. Report: logical, disk, attributed RSS, and the ratio held. ringGiven: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling. When: the ring shape is loaded to 46000 nodes and 46000 relationships, then checkpointed. The child RSS is sampled. Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries. Baseline: logical 29.99 MB – 29.99 MB, disk 1.77 MB – 1.77 MB, attributed RSS 35.30 MB – 35.89 MB, ratio 1.177x–1.197x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes. Validated: logical 29.99 MB, disk 1.77 MB, attributed RSS 36.32 MB, ratio 1.211x. Report: logical, disk, attributed RSS, and the ratio held. dense ringGiven: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling. When: the dense ring shape is loaded to 5000 nodes and 100000 relationships, then checkpointed. The child RSS is sampled. Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries. Baseline: logical 31.41 MB – 31.41 MB, disk 1.62 MB – 1.62 MB, attributed RSS 31.27 MB – 32.46 MB, ratio 0.996x–1.034x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes. Validated: logical 31.41 MB, disk 1.61 MB, attributed RSS 32.21 MB, ratio 1.026x. Report: logical, disk, attributed RSS, and the ratio held. |
|
Loadtest baselines comparison
Loadtest baseline comparisonFrozen baselines vs current reports; bands match the Status: ok Pairs: fraud-loadtest: compared · identity-loadtest: compared fraud-loadtest
Per-use-case p99 latency
identity-loadtest
Per-use-case p99 latency
|
Summary
Optimization: the shortest route's endpoint anchor probes a property-seeked endpoint at the pair bound (16) instead of listing every match first, so a non-selective end map no longer builds a list of millions only to learn the pair product is over the bound (ADR-0555).
Changes
node_candidates_capped,Capped::{Within, Over}); a residual scan stops at the firstcap + 1matches. Uncapped callers are unchanged.Graph::lookup_nodes_upto_at/ExecGraph::lookup_nodes_uptothread the cap to the scan. A view with write scratch answers in full: a truncated base answer could lose members to scratch deletions and read as complete.Route::anchorresolves each endpoint as scan / listed (<= 16) / over. A side is built in full only when it drives a fan (both sides over keeps the shorter-side choice, so 100 ends against 100k starts still fans backward from the 100).Verification
cargo test -p relate-executor(1389 passed),cargo fmt --all --check,cargo clippy --workspace --all-targets --exclude relate-python -- -D warningsstellar-prep.sh localbriefs generated; review passes not run in this sessionFixes LABS-1318.
🤖 Generated with Claude Code