ArcadeDB Graph Data Ingestion — Performance & Issues on HA Cluster

[babasaikiran]

Dataset

• 17,936 nodes (22 vertex types) + 31,832 edges (9 edge types)
• Records are wide: 50+ fields per node
• ArcadeDB v26.4.2, standalone (standalone works fine — cluster has issues)

---

What we do

Step 1: Load & Analyze

• Loads nodes.json and edges.json (flat JSON arrays)
• Groups nodes by node_type → becomes ArcadeDB vertex type (e.g. Switch, VM, vPortgroup)
• Groups edges by relation_type → becomes ArcadeDB edge type (e.g. connected-by, runs)
• Pre-checks for dangling edges (edges referencing node_ids not in the nodes file)

Step 2: Schema Setup

• Creates vertex types and edge types
• Declares node_id property + UNIQUE index on every vertex type
• Declares left_id / right_id properties + NOTUNIQUE indexes on every edge type

Step 3: Insert Nodes

• Bulk insert: INSERT INTO Type CONTENT [{...}, {...}, ...] with batches of 200
• 8 parallel workers
• On bulk failure, falls back to individual inserts per record
• Performance: 17,936 nodes in ~230s (~78 nodes/sec)

Step 4: RID Map Fetch

• Waits 5s for page compaction to settle (required — see issues below)
• Fetches node_id → @rid mapping via SELECT node_id, @rid AS rid FROM Type LIMIT 1000 SKIP N
• Gap-fills any records missed by pagination via individual WHERE node_id = 'xxx' lookups
• Direct RIDs (FROM #12:0 TO #14:7) are much faster than subquery lookups
• Performance: 17,935 RIDs in ~20s

Step 5: Insert Edges

• CREATE EDGE Type FROM #rid TO #rid SET key = val, key = val, ...
• Batches of 50 statements, 8 parallel workers
• Dangling edges (node not in RID map) are skipped and logged
• Performance: 30,604 edges in ~1,250s (~24 edges/sec)

Step 6: Verify & Log

• Counts per type, compares against expected
• All failures logged to JSON file for retry

---

Performance Summary

Step	Records	Time	Rate
Nodes (bulk)	17,936	~230s	~78/sec
RID fetch	17,935	~20s	~900/sec
Edges (individual)	30,604	~1,250s	~24/sec
Total	~48K	~25 min

All of these work fine on standalone — they only fail on the HA cluster.

---

Question

Is there a faster way to ingest edges with properties on an HA cluster?

Currently we're limited to ~24 edges/sec because each edge needs an individual HTTP call. We tried:

• /batch endpoint — returns HTTP 500 (JSONObject[@type] not found)
• Semicolon-joined scripts — /command rejects multi-statement input
• BEGIN/COMMIT wrappers — silently drops edges
• CONTENT {...} on CREATE EDGE — silently drops properties

The only reliable approach we found is individual CREATE EDGE ... SET ... statements, one per HTTP call. For 30K edges on a 3-node HA cluster, this takes ~20 minutes.

Is there a bulk edge creation API, binary protocol, or configuration we're missing that would improve this throughput?

ArcadeDB Graph Data Ingestion — Performance & Issues on HA Cluster

Dataset

• 17,936 nodes (22 vertex types) + 31,832 edges (9 edge types)
• Records are wide: 50+ fields per node
• ArcadeDB v26.4.2, 3-node HA cluster (standalone works fine — cluster has issues)

---

What we do

Step 1: Load & Analyze

• Loads nodes.json and edges.json (flat JSON arrays)
• Groups nodes by node_type → becomes ArcadeDB vertex type (e.g. Switch, VM, vPortgroup)
• Groups edges by relation_type → becomes ArcadeDB edge type (e.g. connected-by, runs)
• Pre-checks for dangling edges (edges referencing node_ids not in the nodes file)

Step 2: Schema Setup

• Creates vertex types and edge types
• Declares node_id property + UNIQUE index on every vertex type
• Declares left_id / right_id properties + NOTUNIQUE indexes on every edge type

Step 3: Insert Nodes

• Bulk insert: INSERT INTO Type CONTENT [{...}, {...}, ...] with batches of 200
• 8 parallel workers
• On bulk failure, falls back to individual inserts per record
• Performance: 17,936 nodes in ~230s (~78 nodes/sec)

Step 4: RID Map Fetch

• Waits 5s for page compaction to settle (required — see issues below)
• Fetches node_id → @rid mapping via SELECT node_id, @rid AS rid FROM Type LIMIT 1000 SKIP N
• Gap-fills any records missed by pagination via individual WHERE node_id = 'xxx' lookups
• Direct RIDs (FROM #12:0 TO #14:7) are much faster than subquery lookups
• Performance: 17,935 RIDs in ~20s

Step 5: Insert Edges

• CREATE EDGE Type FROM #rid TO #rid SET key = val, key = val, ...
• Batches of 50 statements, 8 parallel workers
• Dangling edges (node not in RID map) are skipped and logged
• Performance: 30,604 edges in ~1,250s (~24 edges/sec)

Step 6: Verify & Log

• Counts per type, compares against expected
• All failures logged to JSON file for retry

---

Performance Summary

Step	Records	Time	Rate
Nodes (bulk)	17,936	~230s	~78/sec
RID fetch	17,935	~20s	~900/sec
Edges (individual)	30,604	~1,250s	~24/sec
Total	~48K	~25 min

All of these are on standalone — the ingestion fails directly on the HA cluster.

---

Question

Is there a faster way to ingest edges with properties on an HA cluster?

Currently we're limited to ~24 edges/sec because each edge needs an individual HTTP call. We tried:

• /batch endpoint — returns HTTP 500 (JSONObject[@type] not found)
• Semicolon-joined scripts — /command rejects multi-statement input
• BEGIN/COMMIT wrappers — silently drops edges
• CONTENT {...} on CREATE EDGE — silently drops properties

The only reliable approach we found is individual CREATE EDGE ... SET ... statements, one per HTTP call. For 30K edges on a 3-node HA cluster, this takes ~20 minutes.

Is there a bulk edge creation API, binary protocol, or configuration we're missing that would improve this throughput?

[lvca]

Hi @babasaikiran I fixed the 500 error with the /batch http API. In order to speedup ingestion with a cluster configuration, you have to kill latency. If each write in a cluster of 3 nodes has to wait for at least 2 notes to agree, then it's physically impossible to go faster than this because each insert is waiting for the previous to complete.

But if you use async insertion, you can go much much faster.

The dedicated bulk path is /api/v1/batch/{db} - the same endpoint you got stuck on (now fixed). It's designed exactly for this scenario:

• Single HTTP request, single Raft round for thousands of records (commitEvery=50000 default).
• Uses the internal GraphBatch API: WAL off by default, parallel flush, pre-allocated edge chunks, light edges optional.
• Streams JSONL or CSV - no full-body buffering.
• Vertices first (with @id temp IDs), edges next referencing those IDs or existing #bucket:pos RIDs.

For you use case 17.9K nodes + 30.6K edges that would collapse from ~25 minutes to seconds.

Order of speed-ups, best first:

1. POST /api/v1/batch JSONL - one request for the whole import (or chunks of 100K). Best for HA.
2. language=sqlScript on /command - many SQL statements per request, one transaction = one Raft round. Useful when batch JSONL doesn't fit (mixed DML, schema, etc.).
3. Drop to embedded mode and use db.async().createVertex(...) / Database.async().command(...) - only relevant if they can run in-process. Doesn't apply to a remote HA cluster.
4. awaitResponse=false - marginal; same per-op cost on the server.

Note for HA: every write hits the leader regardless of which node received it (follower forwards), so issuing 30K individual CREATE EDGE calls means 30K Raft rounds. That's the real bottleneck, not the HTTP layer. /batch collapses those into one.

[lvca]

Hi @babasaikiran , did you re-test the insertion again?