BabelDOC is a high-performance document translation engine specifically optimized for scientific PDF translation with layout preservation. It leverages Large Language Models (LLMs) and computer vision to transform complex documents into bilingual or monolingual PDFs while maintaining the original visual structure, including formulas, tables, and multi-column layouts. pyproject.toml1-13 README.md32-35
The project is developed by funstory-ai and serves as the core engine for services like Immersive Translate. It is primarily designed to be embedded as a library but provides a robust CLI for debugging and simple tasks. README.md32-38
OnnxModel) to identify document regions and reconstruct them accurately in the target language. README.md140-155formular-font-pattern) and character pattern matching. babeldoc/main.py171-177WatermarkOutputMode. babeldoc/main.py23 babeldoc/main.py210-214il_version_1) to decouple PDF parsing from rendering, allowing for complex transformations between the original and translated states. README.md32-38 docs/README.md4-8asyncio to handle long-running translation tasks, progress monitoring, and concurrent LLM requests via do_translate_async_stream. babeldoc/main.py1-27 babeldoc/format/pdf/high_level.py20BabelDOC operates as a multi-stage pipeline. It ingests a PDF, decomposes it into an Intermediate Layer (IL), processes the text through LLMs, and reconstructs a new PDF using the translated content.
The following diagram illustrates how the high-level components and CLI entry points relate to the core translation logic.
Sources: babeldoc/main.py67 babeldoc/format/pdf/high_level.py20 pyproject.toml66-67
The core of BabelDOC is an 8-stage pipeline managed by async_translate. This pipeline handles everything from raw byte parsing to final PDF assembly. For large documents, it employs a SplitManager and ResultMerger to process chunks in parallel and recombine them.
The il_version_1 schema acts as the "source of truth". It stores characters (PdfCharacter), styles (PdfStyle), and layout information in a format defined by XSD/RELAX NG schemas, making it easier to manipulate than raw PDF object streams. docs/README.md4-8
BabelDOC uses DocLayout-YOLO (via local OnnxModel) or remote RPC services (RpcDocLayoutModel) to detect document structures like headers, paragraphs, and tables. This ensures the translator understands the reading order and visual boundaries. babeldoc/main.py63-89
The system abstracts LLM interactions through BaseTranslator (implemented by OpenAITranslator). It includes a sophisticated Glossary system using hyperscan for high-speed term matching and an AutomaticTermExtractor to maintain technical consistency. babeldoc/main.py24-25 pyproject.toml49
Scientific PDFs require specific fonts and CMaps. BabelDOC includes an automated asset management system (babeldoc.assets.assets) to download models and a FontMapper to handle font embedding and subsetting across languages. babeldoc/main.py19 pyproject.toml53
BabelDOC includes a NativePageInterpreter (in new_parser) for improved performance over the legacy pdfminer-based frontend, and an ExecutorServer that exposes the engine as a REST API for microservice deployments.
This diagram maps the primary Python classes to their functional roles within the translation lifecycle.
Sources: babeldoc/main.py22-25 babeldoc/main.py32-40 pyproject.toml33
uv, CLI usage, supported languages, and basic Python API examples using do_translate_async_stream. README.md60-111TranslationConfig class and the babeldoc.toml configuration format. babeldoc/main.py22-23 babeldoc/main.py32-40Sources: pyproject.toml1-55 README.md32-50 babeldoc/main.py21-27 babeldoc/format/pdf/high_level.py1-20
Refresh this wiki
