Skip to content

refactor(python): one row-value walk for Record.data()/to_dict() and result_graph (LABS-1474) - #247

Merged
jexp merged 1 commit into
mainfrom
labs-1474-row-value-walk
Nov 12, 2026
Merged

jexp merged 1 commit into
mainfrom
labs-1474-row-value-walk

Conversation

@jexp

@jexp jexp commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Refactor (python surface). record.py::_convert was a second hand-rolled walk over the row-value vocabulary beside result_graph.collect_graph_entities. Both now call one helper, result_graph.map_row_value, so the container set (Mapping, list, tuple, set, frozenset) lives in one place.

  • New neutral module relate/row_values.py owns the vocabulary: map_row_value rebuilds a value (used by Record.data()/to_dict()), iter_graph_values yields graph values without copying (used by collect_graph_entities); both dispatch through one _children helper.
  • record._convert and result_graph._CONTAINER_TYPES removed.

User-facing impact: Record.data()/to_dict() now also convert graph values inside sets and non-dict Mappings (previously left as is). Otherwise unchanged.

Verification

  • python/tests against a maturin-built wheel: 338 passed, 8 skipped
  • cargo fmt --all --check clean; full local stellar round: one medium (fixed) plus nits (pass-through and explicit Path branch fixed)
  • Workspace clippy fails on current main in relate-front (enters_arithmetic_search dead code), unrelated; this change touches no Rust

LABS-1474

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Rust review: no rust-relevant changes (no Rust files modified). ✅

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Security review (security-review skill)

Reviewed the three-file Python refactor that unifies the row-value walk behind a new relate/row_values.py (map_row_value, iter_graph_values) and rewires Record.data() / to_dict() and collect_graph_entities onto it. No authN/authZ, network, file I/O, secrets, dependency, or native-code changes are involved: the walk only reshapes values already returned by the local engine, and the container/leaf vocabulary and recursion behaviour match the code it replaces. Nothing is handed to eval/exec/subprocess, and rebuilt outputs are plain dict/list copies.

No issues found.

Security review: ✅ no issues found

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stellar review round 1: 4 inline finding(s); see summary comment.


A ``Node``, ``Relationship`` or ``Path`` is replaced by
``on_graph_value(value)`` (the callback decides whether to descend into
it); a ``Mapping`` becomes a ``dict``, a list, tuple, set or frozenset a

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[High] structure (flag-logic-wrong-layer): The new shared walk is generic row-value conversion, but it lives in the graph-extraction feature module: record.py — home of the public Record row type — now imports relate.result_graph for it,

Suggested fix: Extract map_row_value and _SEQUENCE_TYPES into a neutral row-value module (or into record.py, with result_graph importing from it) so the dependency points result_graph → record, not the reverse; leave result_graph.py owning only ResultGraph/collect_graph_entities and update its modu

finding F-1-1

walk(item)
collect(entity)
else:
raise TypeError(f"not a graph value: {type(value).__name__}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] branching (std-2): This branch is unreachable: map_row_value invokes the callback only under isinstance(value, (Node, Relationship, Path)), and the recursive calls feed it value.nodes / value.relationships, whic

Suggested fix: Delete the else; the walk owns the vocabulary test, so collect should be only the two store branches plus the Path flatten. If the sink's precondition must be defended, assert it once at the collect_graph_entities boundary instead of adding a branch the sole caller cannot reach.

finding F-1-2

for row in rows:
for value in row.values():
walk(value)
map_row_value(value, collect)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] boundaries (std-4): map_row_value is a mapper — it rebuilds every nested Mapping and sequence — and the collector drops its result, so cursor.graph() / viz.to_visualization_graph now allocate a full converted cop

Suggested fix: Keep the leaf vocabulary ((Node, Relationship, Path)) and _SEQUENCE_TYPES shared, but give the collector an in-place visitor beside the mapper (visit_row_value(value, on_graph_value)) instead of routing it through map_row_value; a sink must not build a copy it discards.

finding F-1-3

collect(entity)
else:
raise TypeError(f"not a graph value: {type(value).__name__}")
return value

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] boundaries (std-5): The walk only invokes the callback for Node, Relationship or Path, so this branch can never fire — a dead raise beside the walk's own handled pass-through (the no-panic-beside-a-handled-form rul

Suggested fix: Type the callback over the graph-value union with no return value, delete the unreachable raise, and derive the vocabulary (walk + callback) from one central definition so a new entity type is an edit in a single place.

finding F-1-4

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Stellar review (stellar-review skill)

Round 4 · head 10cf1a53 · scope: 0/22 units (22 sealed from earlier rounds).

Still open from earlier rounds

  • [Medium] boundaries (std-5) F-2-1: surfaces/relate-python/python/relate/row_values.py:22-30 — The Optional return makes a shared vocabulary helper carry a leaf sentinel that its two callers must interpret with different idioms: map_row_value branches on children is None, while `iter_graph_
  • [Medium] boundaries (std-4) F-2-2: surfaces/relate-python/python/relate/row_values.py:22-30 — _children copies every container's children into a fresh list, so iter_graph_values — documented as the walk that builds nothing, and split from map_row_value precisely so the read path stops re
  • [Medium] boundaries (q-abstraction) F-2-3: surfaces/relate-python/python/relate/result_graph.py:61-70 — The graph walk's path shape knowledge is split from the rest of the vocabulary: row_values._GRAPH_VALUES declares Path a leaf and iter_graph_values therefore yields paths unexpanded, while this

Stellar review: ⚠️ 3 issue(s) open (0 + 0 anchored new (fail/low) + 3 carried)

@jexp
jexp force-pushed the labs-1474-row-value-walk branch from 5cb4d0a to 86583f3 Compare November 12, 2026 15:08

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stellar review round 2: 3 inline finding(s); see summary comment.

return list(value)
return None


Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] boundaries (std-5): The Optional return makes a shared vocabulary helper carry a leaf sentinel that its two callers must interpret with different idioms: map_row_value branches on children is None, while `iter_graph_

Suggested fix: Drop the Optional. Either return the container's own iterable view (value.values() / value) and let each walk's isinstance branches decide leaf-ness (map needs to know container-vs-leaf to rebuild []; iter does not), or delete _children and put the two isinstance arms beside `_GRAPH_VALU

finding F-2-1

return list(value)
return None


Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] boundaries (std-4): _children copies every container's children into a fresh list, so iter_graph_values — documented as the walk that builds nothing, and split from map_row_value precisely so the read path stops re

Suggested fix: Return the container's existing iterable (value.values() for a Mapping, value itself for a sequence) instead of list(...); both call sites only iterate, and map_row_value's comprehensions build the new container anyway. Keep the vocabulary constants as the single source of which types are wa

finding F-2-2

items = value.values() if isinstance(value, Mapping) else value
for item in items:
walk(item)
collect(entity)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] boundaries (q-abstraction): The graph walk's path shape knowledge is split from the rest of the vocabulary: row_values._GRAPH_VALUES declares Path a leaf and iter_graph_values therefore yields paths unexpanded, while this

Suggested fix: Give the entity stream one contract: have the row-values walk flatten a Path into its nodes and relationships (or add iter_graph_entities beside it that does) so Path's internal shape lives with the rest of the vocabulary. collect then becomes a flat classify-and-dedup loop over the yielded

finding F-2-3

@jexp
jexp marked this pull request as ready for review November 12, 2026 18:20
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Spec-harness coverage — 🟡

  • 🟡 spec coverage 99.49% — 172 genuine failures (0.51% of the executed scenarios)
  • scenarios run: 33642 (passed 33470, failed 172)
  • bands: 🟢 zero failures · 🟡 failure share ≤ 10% · 🔴 beyond (ADR-0482 §4 — the light never fails the build)
  • suite runtime: 59s
  • neo4j_expected_fail (not counted): 412
  • nondeterministic (not counted): 2
  • skipped (not counted): 2323
  • harness ref: 1022d60ed6c582f0cf72a75e14e571026feb888e
failure category count
result-mismatch 32
syntax-error 31
error-class-mismatch 29
missing-validation 27
notification-missing 19
semantic-error 10
type-error 10
other 9
setup-failed-syntax 3
result-temporal-fidelity 2

Rerun spec-harness to refresh. Coverage below 100% never fails the build (ADR-0482 §4).

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Loadtest baselines comparison

Warning: loadtest comparison reported problems — see the flagged pairs below.


Loadtest baseline comparison

Frozen baselines vs current reports; bands match the compare-loadtests gate.

Status: warning

Pairs: fraud-loadtest: compared · identity-loadtest: compared (failing rows)

fraud-loadtest

baselines/fraud-loadtest.json → target/fraud-loadtest.json

Metric Baseline PR Δ Status
environment relate-loadtest-baseline/v1 relate-loadtest-baseline/v1 — match
errors 0 0 +0 ok
ops/s 3938.56 5408.33 +1469.77 (+37.3%) ok
p99 (worst: activity_page) 7.48 ms 6.99 ms -0.50 ms (-6.6%) ok
logical bytes 17,344,481 16,362,171 -982,310 (-5.7%) ok
Per-use-case p99 latency
Use case ops Baseline p99 PR p99 Δ Status
p2p_transfer 3,855 8.04 ms 7.19 ms -0.85 ms (-10.6%) ok
payroll_scatter 1,089 17.85 ms 10.70 ms -7.16 ms (-40.1%) ok
onboard_customer 801 8.53 ms 6.68 ms -1.85 ms (-21.7%) ok
mule_scatter 811 14.80 ms 9.68 ms -5.12 ms (-34.6%) ok
layering_cycle 819 15.10 ms 9.44 ms -5.66 ms (-37.5%) ok
recent_transfers 2,306 7.65 ms 6.74 ms -0.92 ms (-12.0%) ok
customer_portfolio 1,418 7.37 ms 6.38 ms -1.00 ms (-13.5%) ok
outgoing_volume 1,093 7.48 ms 6.04 ms -1.44 ms (-19.3%) ok
detect_fanout 842 25.78 ms 10.70 ms -15.07 ms (-58.5%) ok
detect_cycle 849 7.55 ms 6.02 ms -1.53 ms (-20.2%) ok
activity_page 1,138 7.48 ms 6.99 ms -0.50 ms (-6.6%) ok
search_customer 796 7.84 ms 6.97 ms -0.88 ms (-11.2%) ok
counterparties 1,103 7.45 ms 6.23 ms -1.22 ms (-16.4%) ok
payment_path 876 7.47 ms 6.68 ms -0.79 ms (-10.5%) ok
amount_band 831 7.30 ms 6.34 ms -0.96 ms (-13.1%) ok
flag_account 697 8.21 ms 6.56 ms -1.65 ms (-20.1%) ok
reverse_last_out 676 8.04 ms 6.87 ms -1.17 ms (-14.5%) ok

identity-loadtest

baselines/identity-loadtest.json → target/identity-loadtest.json

Metric Baseline PR Δ Status
environment relate-loadtest-baseline/v1 relate-loadtest-baseline/v1 — match
errors 0 2 +2 fail (2 errors)
ops/s 573.51 924.51 +351.00 (+61.2%) ok
p99 (worst: group_roster) 7.72 ms 7.77 ms +0.06 ms (+0.7%) ok
logical bytes 7,167,614 7,391,917 +224,303 (+3.1%) ok
Per-use-case p99 latency
Use case ops Baseline p99 PR p99 Δ Status
provision_user 1,373 11.00 ms 8.40 ms -2.60 ms (-23.6%) ok
add_to_group 1,775 18.57 ms 12.86 ms -5.71 ms (-30.7%) ok
grant_role 1,079 18.52 ms 11.52 ms -7.00 ms (-37.8%) ok
attach_device 907 18.88 ms 12.02 ms -6.85 ms (-36.3%) ok
nested_grant 736 19.41 ms 11.73 ms -7.68 ms (-39.6%) ok
effective_access 2,173 25.25 ms 16.24 ms -9.00 ms (-35.7%) ok
org_walk 1,387 19.94 ms 12.89 ms -7.05 ms (-35.4%) ok
optional_manager 1,312 20.00 ms 13.68 ms -6.32 ms (-31.6%) ok
detect_nested_admin 810 19.97 ms 12.00 ms -7.98 ms (-39.9%) ok
detect_sod 817 19.60 ms 11.87 ms -7.73 ms (-39.5%) ok
stale_grants 827 15.06 ms 11.12 ms -3.94 ms (-26.2%) ok
search_directory 1,147 10.91 ms 9.09 ms -1.82 ms (-16.7%) ok
group_roster 1,047 7.72 ms 7.77 ms +0.06 ms (+0.7%) ok
access_review 801 20.02 ms 11.87 ms -8.15 ms (-40.7%) ok
reporting_path 794 30.98 ms 18.00 ms -12.98 ms (-41.9%) ok
disable_user 644 19.50 ms 12.60 ms -6.90 ms (-35.4%) ok
add_alias 710 19.99 ms 11.37 ms -8.62 ms (-43.1%) ok
revoke_membership 674 8.10 ms 7.03 ms -1.07 ms (-13.2%) ok
detach_device 538 14.79 ms 9.63 ms -5.17 ms (-34.9%) ok
clear_expiry 449 17.39 ms 11.01 ms -6.38 ms (-36.7%) ok

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Memory shape

Note: loadtest comparison completed with no failing rows.


Memory shape

Status: ok

All memory shapes held. 🟢

base

Given: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling.

When: the base shape is loaded to 0 nodes and 0 relationships, then checkpointed. The child RSS is sampled.

Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries.

Baseline: logical 0.00 MB – 0.00 MB, disk 0.00 MB – 0.00 MB, attributed RSS 0.00 MB – 0.00 MB, ratio 0.000x–0.000x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes.

Validated: logical 0.00 MB, disk 0.00 MB, attributed RSS 0.00 MB, ratio 0.000x.

Report: logical, disk, attributed RSS, and the ratio held.

flat

Given: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling.

When: the flat shape is loaded to 120000 nodes and 0 relationships, then checkpointed. The child RSS is sampled.

Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries.

Baseline: logical 26.02 MB – 26.02 MB, disk 1.95 MB – 1.95 MB, attributed RSS 36.49 MB – 36.93 MB, ratio 1.402x–1.419x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes.

Validated: logical 26.02 MB, disk 1.26 MB, attributed RSS 37.65 MB, ratio 1.447x.

Report: logical, disk, attributed RSS, and the ratio held.

fanout

Given: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling.

When: the fanout shape is loaded to 52016 nodes and 52000 relationships, then checkpointed. The child RSS is sampled.

Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries.

Baseline: logical 28.62 MB – 28.62 MB, disk 1.75 MB – 1.75 MB, attributed RSS 34.93 MB – 36.61 MB, ratio 1.220x–1.279x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes.

Validated: logical 27.43 MB, disk 1.10 MB, attributed RSS 36.01 MB, ratio 1.313x.

Report: logical, disk, attributed RSS, and the ratio held.

ring

Given: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling.

When: the ring shape is loaded to 46000 nodes and 46000 relationships, then checkpointed. The child RSS is sampled.

Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries.

Baseline: logical 29.99 MB – 29.99 MB, disk 1.77 MB – 1.77 MB, attributed RSS 35.30 MB – 35.89 MB, ratio 1.177x–1.197x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes.

Validated: logical 29.99 MB, disk 1.20 MB, attributed RSS 35.86 MB, ratio 1.196x.

Report: logical, disk, attributed RSS, and the ratio held.

dense ring

Given: an empty relate serve, 1 integer properties on every node, text indexes 0, point indexes 0, range indexes 0, and a 32.00 MB logical ceiling.

When: the dense ring shape is loaded to 5000 nodes and 100000 relationships, then checkpointed. The child RSS is sampled.

Then: the database holds those counts. The load finishes under the ceiling. Logical bytes, disk, attributed RSS, and the ratio stay inside the frozen boundaries.

Baseline: logical 31.41 MB – 31.41 MB, disk 1.62 MB – 1.62 MB, attributed RSS 31.27 MB – 32.46 MB, ratio 0.996x–1.034x. A reading may sit 5% outside the ratio, and 15% or 1.00 MB outside the sizes.

Validated: logical 31.11 MB, disk 0.73 MB, attributed RSS 31.95 MB, ratio 1.027x.

Report: logical, disk, attributed RSS, and the ratio held.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Memory controls and workspace memory calibration

Warning: the comparison document target/memory-controls.md is missing — the render step produced nothing this run.


(comparison document unavailable)

…result_graph (LABS-1474)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jexp
jexp force-pushed the labs-1474-row-value-walk branch from 86583f3 to 10cf1a5 Compare November 12, 2026 19:24
@oplane-bot

oplane-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

Oplane Security Review

Oplane reviewed this PR and did not identify any security requirements that need attention. Review the generated threat model.


Change comment settings · Set up Oplane in your agent · Workflow guidance

@jexp
jexp merged commit 45677e7 into main Nov 12, 2026
53 of 55 checks passed
@jexp
jexp deleted the labs-1474-row-value-walk branch November 12, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant