When an AI retrieval-augmented generation (RAG) pipeline hallucinates on a question sourced from a scientific paper or financial report, the debugging process almost invariably looks at the wrong layer. Engineers test alternative embedding models, tweak chunk token budgets, or introduce expensive cross-encoder rerankers.
Yet in a surprising fraction of failure cases, the corruption happens before embedding: in the text extraction layer where the parser attempts to reconstruct human reading order from a format that was never designed to store it.
What a PDF File Actually Stores
A PDF is essentially a set of rendering commands designed to place ink or pixels consistently across any screen or printer. Under the hood, text lives inside stream operators bracketed by BT (Begin Text) and ET (End Text):
BT
/F1 11 Tf
72 700 Td
[(Retrieval) -250 (augmented) -250 (generation)] TJ
0 -14 Td
(depends on document layout.) Tj
ET
Here, Td moves the coordinate cursor, Tj outputs raw glyph codes, and TJ outputs strings separated by numerical kerning shifts. Crucially:
- There is no concept of a "sentence" or "paragraph" in an untagged PDF stream.
- The order of rendering commands in the file reflects the sequence in which the generator tool drew elementsânot necessarily how a human reads them.
- Kerning offsets (like the
-250in the example above) are horizontal spacing nudges, leaving the parser to guess where word spaces belong.
When you pass this document to an automated parser, the tool must guess which glyphs form words, which words form lines, and in what order those lines should be concatenated.
The Multi-Column Braiding Problem
The most damaging extraction error occurs in two-column or multi-column layoutsâthe standard format for arXiv papers, IEEE publications, and corporate annual reports.
When an extraction library attempts to "fix" raw stream order by sorting text visually by vertical position ($y$-coordinate), it reads across both columns simultaneously. Line 1 of column A is followed immediately by line 1 of column B, then line 2 of column A, then line 2 of column B:
Column A: Column B:
Rats were fed a All authors declare
standard chow diet no competing
for fourteen days. financial interests.
Result of naive Y-coordinate sorting:
"Rats were fed a All authors declare standard chow diet no competing for fourteen days. financial interests."
This resulting text is insidious because it never throws a Python exception, passes character-count heuristics, and retains plausible English vocabulary. But when this chunk is sliced into a 512-token window and passed to an embedding model, the vector represents a semantic blend of laboratory protocol and financial disclosures. The vector sits in embedding space near neither topic, making accurate retrieval mathematically impossible.
Three Practical Tiers to Recover Reading Order
Depending on your pipeline constraints and throughput requirements, here are three practical engineering approaches for handling multi-column documents:
Tier 1: Geometric Block Partitioning with Vertical Banding
If you run lightweight CPU-only ingestion using libraries like PyMuPDF (fitz), you can operate on layout bounding boxes (blocks) rather than flat character dumps.
A common naive heuristic splits the page at the horizontal midpoint, but that fails when full-width titles, abstracts, or wide tables appear in the middle of a page. A more robust approach groups blocks into vertical bands separated by spanning elements:
# Practical 2-column reading order parser with vertical band guards
def extract_two_column_page(page, span_threshold=0.75):
"""
Extracts text from a two-column page while preserving reading sequence.
Groups text into vertical bands whenever a full-width block appears
(e.g., page title, abstract, or full-width table/figure), preventing
mid-page spanning elements from corrupting the sequence.
"""
# PyMuPDF block format: (x0, y0, x1, y1, text, block_no, block_type)
# block_type == 0 represents text blocks
text_blocks = [b for b in page.get_text("blocks") if b[6] == 0]
if not text_blocks:
return ""
page_width = page.rect.width
midpoint = page_width / 2.0
# Sort blocks by top coordinate to process page bands top-to-bottom
text_blocks.sort(key=lambda b: b[1])
ordered_blocks = []
current_left = []
current_right = []
def flush_columns():
nonlocal current_left, current_right, ordered_blocks
if current_left:
ordered_blocks.extend(sorted(current_left, key=lambda b: b[1]))
current_left = []
if current_right:
ordered_blocks.extend(sorted(current_right, key=lambda b: b[1]))
current_right = []
for b in text_blocks:
x0, y0, x1, y1 = b[0], b[1], b[2], b[3]
box_width = x1 - x0
# Spanning elements act as horizontal barriers between column bands
if box_width >= page_width * span_threshold:
flush_columns()
ordered_blocks.append(b)
elif x0 < midpoint:
current_left.append(b)
else:
current_right.append(b)
flush_columns()
return "\n\n".join(b[4].strip() for b in ordered_blocks if b[4].strip())
Note: In production environments with varying margins or 3-column formats, dynamic gutter detection via horizontal projection profiles replaces the static midpoint.
Tier 2: Layout-Aware Vision and Segmentation Models
When documents include irregular multi-column grids, floating callout boxes, and complex multi-row tables, geometric heuristics begin to accumulate edge-case failures.
Layout-aware vision models (such as Docling, Marker, or layout segmentation vision transformers) treat the rendered page as a visual artifact. These architectures predict semantic regions (headings, captions, body paragraphs, tables) and explicitly infer the reading order graph between bounding boxes before converting elements into structured Markdown.
This approach requires significantly higher compute (typically GPU acceleration), but it prevents table rows from collapsing into unstructured text strings.
Tier 3: Producer-Side Tagged PDFs
If your team produces the documents entering the RAG corpus (e.g., internal research memos or documentation exports), the cleanest long-term solution is emitting Tagged PDF (PDF/UA or ISO 32000 tagged structures).
Tagged PDFs embed a structural logical tree directly into the file container, explicitly specifying reading sequence regardless of visual layout positioning. This eliminates heuristic guesswork for both screen readers and downstream AI extractors.
Audit Checklist Before Benchmarking Embedding Models
Before spending budget on new embedding endpoints or vector database re-indexing, run this quick quality audit on your raw text chunks:
- Spot-check multi-column transitions: Extract 20 random pages from multi-column documents and check whether sentences jump horizontally across columns.
- Inspect table headers: Confirm whether numbers in parsed output remain linked to their column and row descriptors or have flattened into meaningless lists of floats.
- Verify formula integrity: Check whether mathematical symbols and sub/superscripts maintain syntactic continuity or fragment across line breaks.
- Compare coordinate-sorted vs. raw output: A substantial divergence in text sequence often highlights where layout heuristics are struggling.
Clean document structure upstream is the most reliable foundation for downstream retrieval accuracy.
Update (2026-11-07): Gutter-Crossing Headings & Uncertainty Flags
Following reader feedback regarding centered subheadings and auditability, two specific enhancements address mid-page column anomalies:
-
Gutter-Crossing Intersection: A centered section heading occupying only 35â45% of page width falls below standard full-width thresholds (such as 0.75), yet conceptually spans both columns. Testing whether the bounding box intersects the central column gutter (
x0 < midpoint < x1) accurately identifies it as a spanning separator without misassigning it to the left column. - Exposing Layout Confidence Metadata: In production ingestion, emitting an uncertainty flag for blocks that graze gutter margins or border ambiguous vertical bands allows downstream audits to flag potentially misordered chunks before embedding and indexing.
Top comments (4)
The vertical-band guard is a useful improvement over a single left-then-right pass. One fixture I would add is a centered heading that crosses the column gutter but occupies only 40% of the page width. It is a spanning element semantically, yet the 0.75 width threshold would classify it as an ordinary left-column block.
Testing intersection with the detected gutter, alongside width, would catch that case. I would also expose uncertain block assignments in the extraction output so the audit can distinguish a confidently ordered page from one that only happened to produce plausible text.
Checking intersection with the detected gutter x-span is definitely the cleaner trigger here. A narrow centered heading or short callout breaks the fixed width threshold every time, even though its bounding box clearly bridges the column divider.
Exposing an uncertainty or fallback flag in the block metadata is also essential. For downstream RAG ingestion, knowing whether reading order was resolved deterministically or by a heuristic tie-breaker makes automated quarantine or targeted review feasible instead of debugging silent chunk hallucinations later.
A block-level flag would make that quarantine practical without discarding the whole page. A centered callout can be ambiguous while the two surrounding column runs have a clear order. Marking the entire page uncertain would hide that distinction.
I would retain the affected block identifiers and the ordering decision that used the tie-breaker, then test a page with one ambiguous callout among otherwise straightforward paragraphs. Downstream ingestion could review that local boundary while keeping the confidently ordered material available.
Preserving local block IDs and the specific tie-breaker decision in metadata keeps the audit trail actionable. When an ambiguous callout spans two clear text columns, downstream ingestion can quarantine just that local reading boundary for review instead of dropping valid paragraphs.
In our benchmark runs on multi-column papers, passing the bounding box coordinates alongside the tie-breaker let the chunker isolate the uncertain block into its own standalone element. The surrounding columns still flow in standard reading order, which prevents retrieval gaps in RAG without hallucinating an arbitrary reading sequence.