DEV Community

KolmoPDF
KolmoPDF

Posted on Edited on

Why Naive Text Extraction Destroys RAG on Multi-Column PDFs (and How to Fix Reading Order)

When an AI retrieval-augmented generation (RAG) pipeline hallucinates on a question sourced from a scientific paper or financial report, the debugging process almost invariably looks at the wrong layer. Engineers test alternative embedding models, tweak chunk token budgets, or introduce expensive cross-encoder rerankers.

Yet in a surprising fraction of failure cases, the corruption happens before embedding: in the text extraction layer where the parser attempts to reconstruct human reading order from a format that was never designed to store it.

What a PDF File Actually Stores

A PDF is essentially a set of rendering commands designed to place ink or pixels consistently across any screen or printer. Under the hood, text lives inside stream operators bracketed by BT (Begin Text) and ET (End Text):

BT
  /F1 11 Tf
  72 700 Td
  [(Retrieval) -250 (augmented) -250 (generation)] TJ
  0 -14 Td
  (depends on document layout.) Tj
ET
Enter fullscreen mode Exit fullscreen mode

Here, Td moves the coordinate cursor, Tj outputs raw glyph codes, and TJ outputs strings separated by numerical kerning shifts. Crucially:

  1. There is no concept of a "sentence" or "paragraph" in an untagged PDF stream.
  2. The order of rendering commands in the file reflects the sequence in which the generator tool drew elements—not necessarily how a human reads them.
  3. Kerning offsets (like the -250 in the example above) are horizontal spacing nudges, leaving the parser to guess where word spaces belong.

When you pass this document to an automated parser, the tool must guess which glyphs form words, which words form lines, and in what order those lines should be concatenated.

The Multi-Column Braiding Problem

The most damaging extraction error occurs in two-column or multi-column layouts—the standard format for arXiv papers, IEEE publications, and corporate annual reports.

When an extraction library attempts to "fix" raw stream order by sorting text visually by vertical position ($y$-coordinate), it reads across both columns simultaneously. Line 1 of column A is followed immediately by line 1 of column B, then line 2 of column A, then line 2 of column B:

Column A:                          Column B:
Rats were fed a                    All authors declare
standard chow diet                 no competing
for fourteen days.                 financial interests.

Result of naive Y-coordinate sorting:
"Rats were fed a All authors declare standard chow diet no competing for fourteen days. financial interests."
Enter fullscreen mode Exit fullscreen mode

This resulting text is insidious because it never throws a Python exception, passes character-count heuristics, and retains plausible English vocabulary. But when this chunk is sliced into a 512-token window and passed to an embedding model, the vector represents a semantic blend of laboratory protocol and financial disclosures. The vector sits in embedding space near neither topic, making accurate retrieval mathematically impossible.

Three Practical Tiers to Recover Reading Order

Depending on your pipeline constraints and throughput requirements, here are three practical engineering approaches for handling multi-column documents:

Tier 1: Geometric Block Partitioning with Vertical Banding

If you run lightweight CPU-only ingestion using libraries like PyMuPDF (fitz), you can operate on layout bounding boxes (blocks) rather than flat character dumps.

A common naive heuristic splits the page at the horizontal midpoint, but that fails when full-width titles, abstracts, or wide tables appear in the middle of a page. A more robust approach groups blocks into vertical bands separated by spanning elements:

# Practical 2-column reading order parser with vertical band guards
def extract_two_column_page(page, span_threshold=0.75):
    """
    Extracts text from a two-column page while preserving reading sequence.
    Groups text into vertical bands whenever a full-width block appears
    (e.g., page title, abstract, or full-width table/figure), preventing
    mid-page spanning elements from corrupting the sequence.
    """
    # PyMuPDF block format: (x0, y0, x1, y1, text, block_no, block_type)
    # block_type == 0 represents text blocks
    text_blocks = [b for b in page.get_text("blocks") if b[6] == 0]
    if not text_blocks:
        return ""

    page_width = page.rect.width
    midpoint = page_width / 2.0

    # Sort blocks by top coordinate to process page bands top-to-bottom
    text_blocks.sort(key=lambda b: b[1])

    ordered_blocks = []
    current_left = []
    current_right = []

    def flush_columns():
        nonlocal current_left, current_right, ordered_blocks
        if current_left:
            ordered_blocks.extend(sorted(current_left, key=lambda b: b[1]))
            current_left = []
        if current_right:
            ordered_blocks.extend(sorted(current_right, key=lambda b: b[1]))
            current_right = []

    for b in text_blocks:
        x0, y0, x1, y1 = b[0], b[1], b[2], b[3]
        box_width = x1 - x0

        # Spanning elements act as horizontal barriers between column bands
        if box_width >= page_width * span_threshold:
            flush_columns()
            ordered_blocks.append(b)
        elif x0 < midpoint:
            current_left.append(b)
        else:
            current_right.append(b)

    flush_columns()

    return "\n\n".join(b[4].strip() for b in ordered_blocks if b[4].strip())
Enter fullscreen mode Exit fullscreen mode

Note: In production environments with varying margins or 3-column formats, dynamic gutter detection via horizontal projection profiles replaces the static midpoint.

Tier 2: Layout-Aware Vision and Segmentation Models

When documents include irregular multi-column grids, floating callout boxes, and complex multi-row tables, geometric heuristics begin to accumulate edge-case failures.

Layout-aware vision models (such as Docling, Marker, or layout segmentation vision transformers) treat the rendered page as a visual artifact. These architectures predict semantic regions (headings, captions, body paragraphs, tables) and explicitly infer the reading order graph between bounding boxes before converting elements into structured Markdown.

This approach requires significantly higher compute (typically GPU acceleration), but it prevents table rows from collapsing into unstructured text strings.

Tier 3: Producer-Side Tagged PDFs

If your team produces the documents entering the RAG corpus (e.g., internal research memos or documentation exports), the cleanest long-term solution is emitting Tagged PDF (PDF/UA or ISO 32000 tagged structures).

Tagged PDFs embed a structural logical tree directly into the file container, explicitly specifying reading sequence regardless of visual layout positioning. This eliminates heuristic guesswork for both screen readers and downstream AI extractors.

Audit Checklist Before Benchmarking Embedding Models

Before spending budget on new embedding endpoints or vector database re-indexing, run this quick quality audit on your raw text chunks:

  1. Spot-check multi-column transitions: Extract 20 random pages from multi-column documents and check whether sentences jump horizontally across columns.
  2. Inspect table headers: Confirm whether numbers in parsed output remain linked to their column and row descriptors or have flattened into meaningless lists of floats.
  3. Verify formula integrity: Check whether mathematical symbols and sub/superscripts maintain syntactic continuity or fragment across line breaks.
  4. Compare coordinate-sorted vs. raw output: A substantial divergence in text sequence often highlights where layout heuristics are struggling.

Clean document structure upstream is the most reliable foundation for downstream retrieval accuracy.

Update (2026-11-07): Gutter-Crossing Headings & Uncertainty Flags

Following reader feedback regarding centered subheadings and auditability, two specific enhancements address mid-page column anomalies:

  1. Gutter-Crossing Intersection: A centered section heading occupying only 35–45% of page width falls below standard full-width thresholds (such as 0.75), yet conceptually spans both columns. Testing whether the bounding box intersects the central column gutter (x0 < midpoint < x1) accurately identifies it as a spanning separator without misassigning it to the left column.
  2. Exposing Layout Confidence Metadata: In production ingestion, emitting an uncertainty flag for blocks that graze gutter margins or border ambiguous vertical bands allows downstream audits to flag potentially misordered chunks before embedding and indexing.

Top comments (4)

Collapse
 
ahmetozel profile image
Ahmet Özel •

The vertical-band guard is a useful improvement over a single left-then-right pass. One fixture I would add is a centered heading that crosses the column gutter but occupies only 40% of the page width. It is a spanning element semantically, yet the 0.75 width threshold would classify it as an ordinary left-column block.

Testing intersection with the detected gutter, alongside width, would catch that case. I would also expose uncertain block assignments in the extraction output so the audit can distinguish a confidently ordered page from one that only happened to produce plausible text.

Collapse
 
kolmopdf profile image
KolmoPDF •

Checking intersection with the detected gutter x-span is definitely the cleaner trigger here. A narrow centered heading or short callout breaks the fixed width threshold every time, even though its bounding box clearly bridges the column divider.

Exposing an uncertainty or fallback flag in the block metadata is also essential. For downstream RAG ingestion, knowing whether reading order was resolved deterministically or by a heuristic tie-breaker makes automated quarantine or targeted review feasible instead of debugging silent chunk hallucinations later.

Collapse
 
ahmetozel profile image
Ahmet Özel •

A block-level flag would make that quarantine practical without discarding the whole page. A centered callout can be ambiguous while the two surrounding column runs have a clear order. Marking the entire page uncertain would hide that distinction.

I would retain the affected block identifiers and the ordering decision that used the tie-breaker, then test a page with one ambiguous callout among otherwise straightforward paragraphs. Downstream ingestion could review that local boundary while keeping the confidently ordered material available.

Thread Thread
 
kolmopdf profile image
KolmoPDF •

Preserving local block IDs and the specific tie-breaker decision in metadata keeps the audit trail actionable. When an ambiguous callout spans two clear text columns, downstream ingestion can quarantine just that local reading boundary for review instead of dropping valid paragraphs.

In our benchmark runs on multi-column papers, passing the bounding box coordinates alongside the tie-breaker let the chunker isolate the uncertain block into its own standalone element. The surrounding columns still flow in standard reading order, which prevents retrieval gaps in RAG without hallucinating an arbitrary reading sequence.