End-to-End Scientific Review Table Digitization Pipeline

Tabulus is a modular framework for digitizing scientific review tables into structured, citation-aware data. It starts from scientific PDFs, reconstructs reference-containing tables, extracts the paper bibliography, and resolves table citations through inspectable CLI steps and filesystem artifacts.

The rebuilt library implements a complete seven-step persisted workflow, ending in deterministic resolved CSV export. A single monolithic tabulus run orchestration command remains future convenience work.

• PDF profiling and table detection with MinerU.
• Canonical table crops for reproducible adapter comparisons.
• Table reconstruction through a registry of OCR, table-structure, and document vision-language-model adapters.
• Deterministic reference-table classification and table-cell-to-bibliography matching.
• GROBID-backed bibliography extraction from the original PDF.
• Conservative Step 6 scholarly reference resolution using deterministic evidence, Crossref/CORE metadata, and bounded LLM adjudication.
• Deterministic Step 7 resolved CSV export with optional safe continuation merging and physical-table provenance.
• Native table-reconstruction evaluation with Relative Mapping Similarity (RMS).

PDF
 ├─ Step 1: profile PDF and export canonical table crops
 │    └─ Step 2: reconstruct physical tables
 │         └─ Step 3: classify reference-containing tables
 └─ Step 4: extract bibliography from the original PDF

Step 3 selected tables + Step 4 bibliography
 └─ Step 5: match table citations to bibliography positions
      └─ Step 6: resolve paper-level scholarly identities
           └─ Step 7: export resolved physical CSVs
                └─ optional safe continuation merging

Step 4 is a parallel PDF-level branch. Step 6 resolves each linked bibliography entry once per paper, not once per table cell or reconstruction adapter. Step 7 joins those paper-level decisions back onto physical table rows without modifying Step 2 prediction CSVs.

git clone https://github.com/sciknoworg/tabulus.git
cd tabulus
python -m pip install -e ".[dev]"
tabulus --help

For environment-specific setup, see the Windows/CPU, GPU server, and Python library installation guides.

PDF="/path/to/paper.pdf"
ARTIFACT_ROOT="/path/to/tabulus-artifacts/paper"
CROP_ROOT="/path/to/tabulus-output/table-crops/paper"
RECONSTRUCTION="$CROP_ROOT/reconstructions/tesseract-tatr"

# Step 1: PDF profiling and canonical crop export
tabulus profile --pdf "$PDF" --backend pipeline --method auto

# Step 2: table reconstruction
tabulus reconstruct-tables \
  --crops "$CROP_ROOT" \
  --adapter tesseract-tatr \
  --device cpu

# Step 3: reference-table classification
tabulus classify-reference-tables \
  --reconstruction "$RECONSTRUCTION"

# Step 4: bibliography extraction
tabulus extract-bibliography \
  --pdf "$PDF" \
  --out "$ARTIFACT_ROOT" \
  --grobid-url http://localhost:8070

# Step 5: table-cell to bibliography-position matching
tabulus match-references \
  --selected "$RECONSTRUCTION/selected_reference_tables.json" \
  --bibliography "$ARTIFACT_ROOT/references/bibliography.json"

# Step 6: paper-level scholarly reference resolution
tabulus resolve-references \
  --bibliography "$ARTIFACT_ROOT/references/bibliography.json" \
  --reference-matches "$RECONSTRUCTION/references/reference_matches.json" \
  --out "$ARTIFACT_ROOT"

# Step 7: deterministic resolved CSV export
tabulus export-resolved-csv \
  --reference-matches "$RECONSTRUCTION/references/reference_matches.json" \
  --reference-resolution "$ARTIFACT_ROOT/references/reference_resolution.json"

Step 6 requires Crossref, CORE, and OpenAI-compatible LLM configuration. The full option list and credential handling are documented in the Step 6 tutorial.

Step 7 is offline and deterministic. Add --merge-continuations to request safe logical merging while retaining physical resolved CSVs.

See the Data Contracts for exact schemas and filesystem layout.

TabulusBench is the companion benchmark dataset used by the tutorials and evaluation documentation. It contains scientific PDFs, annotated reference-containing table crops, table gold CSVs, and bibliography gold for a subset of papers.

• TabulusBench on Zenodo

The documentation uses P4 as the canonical one-paper worked example. Benchmark gold material is read-only; Tabulus runs should write their generated artifacts outside the gold directories.

python -m pip install -e ".[dev]"
pytest
git diff --check

The supported public library lives in src/tabulus. Retained legacy and research directories are preserved for provenance, but they are not the current public API unless the documentation says so explicitly.

Citation information will be added after the associated thesis or software release is published.

Tabulus is released under the MIT License.