Docling simplifies document processing, parsing diverse formats — including advanced PDF understanding — and providing seamless integrations with the gen AI ecosystem.
Go to file
Gabe Goodhart c605edd8e9
feat: OllamaVlmModel for Granite Vision 3.2 (#1337)
* build: Add ollama sdk dependency

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat: Add option plumbing for OllamaVlmOptions in pipeline_options

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat: Full implementation of OllamaVlmModel

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* feat: Connect "granite_vision_ollama" pipeline option to CLI

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* Revert "build: Add ollama sdk dependency"

After consideration, we're going to use the generic OpenAI API instead
of the Ollama-specific API to avoid duplicate work.

This reverts commit bc6b366468cdd66b52540aac9c7d8b584ab48ad0.

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* refactor: Move OpenAI API call logic into utils.utils

This will allow reuse of this logic in a generic VLM model

NOTE: There is a subtle change here in the ordering of the text prompt and
the image in the call to the OpenAI API. When run against Ollama, this
ordering makes a big difference. If the prompt comes before the image, the
result is terse and not usable whereas the prompt coming after the image
works as expected and matches the non-OpenAI chat API.

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* refactor: Refactor from Ollama SDK to generic OpenAI API

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* fix: Linting, formatting, and bug fixes

The one bug fix was in the timeout arg to openai_image_request. Otherwise,
this is all style changes to get MyPy and black passing cleanly.

Branch: OllamaVlmModel

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>

* remove model from download enum

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* generalize input args for other API providers

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* rename and refactor

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add example

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* require flag for remote services

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* disable example from CI

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add examples to docs

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
2025-04-10 18:03:04 +02:00
.actor feat(actor): Docling Actor on Apify infrastructure (#875) 2025-03-18 10:17:44 +01:00
.github feat: OllamaVlmModel for Granite Vision 3.2 (#1337) 2025-04-10 18:03:04 +02:00
docling feat: OllamaVlmModel for Granite Vision 3.2 (#1337) 2025-04-10 18:03:04 +02:00
docs feat: OllamaVlmModel for Granite Vision 3.2 (#1337) 2025-04-10 18:03:04 +02:00
tests fix(docx): Adding new latex symbols, simplifying how equations are added to text (#1295) 2025-04-08 17:11:37 +02:00
.gitignore ci: Add Github Actions (#4) 2024-07-16 13:05:04 +02:00
.pre-commit-config.yaml feat!: Docling v2 (#117) 2024-10-16 21:02:03 +02:00
CHANGELOG.md chore: bump version to 2.29.0 [skip ci] 2025-04-10 12:24:09 +00:00
CITATION.cff chore: add downloads in README, security policy and update ci actions (#401) 2024-11-21 13:59:45 +01:00
CODE_OF_CONDUCT.md docs: Linux Foundation AI & Data (#1183) 2025-03-19 09:05:57 +01:00
CONTRIBUTING.md docs: Linux Foundation AI & Data (#1183) 2025-03-19 09:05:57 +01:00
Dockerfile chore: properly clean up apt temporary files in Dockerfile (#1223) 2025-03-25 11:10:09 +01:00
LICENSE chore: fix placeholders in license (#63) 2024-09-06 17:10:07 +02:00
MAINTAINERS.md docs: Linux Foundation AI & Data (#1183) 2025-03-19 09:05:57 +01:00
mkdocs.yml feat: OllamaVlmModel for Granite Vision 3.2 (#1337) 2025-04-10 18:03:04 +02:00
poetry.lock chore: update lock file (#1315) 2025-04-07 17:47:51 +02:00
pyproject.toml chore: bump version to 2.29.0 [skip ci] 2025-04-10 12:24:09 +00:00
README.md feat(SmolDocling): Support MLX acceleration in VLM pipeline (#1199) 2025-03-19 15:38:54 +01:00

Docling

Docling

DS4SD%2Fdocling | Trendshift

arXiv Docs PyPI version PyPI - Python Version Poetry Code style: black Imports: isort Pydantic v2 pre-commit License MIT PyPI Downloads Docling Actor LF AI & Data

Docling simplifies document processing, parsing diverse formats — including advanced PDF understanding — and providing seamless integrations with the gen AI ecosystem.

Features

  • 🗂️ Parsing of multiple document formats incl. PDF, DOCX, XLSX, HTML, images, and more
  • 📑 Advanced PDF understanding incl. page layout, reading order, table structure, code, formulas, image classification, and more
  • 🧬 Unified, expressive DoclingDocument representation format
  • ↪️ Various export formats and options, including Markdown, HTML, and lossless JSON
  • 🔒 Local execution capabilities for sensitive data and air-gapped environments
  • 🤖 Plug-and-play integrations incl. LangChain, LlamaIndex, Crew AI & Haystack for agentic AI
  • 🔍 Extensive OCR support for scanned PDFs and images
  • 🥚 Support of Visual Language Models (SmolDocling) 🆕
  • 💻 Simple and convenient CLI

Coming soon

  • 📝 Metadata extraction, including title, authors, references & language
  • 📝 Chart understanding (Barchart, Piechart, LinePlot, etc)
  • 📝 Complex chemistry understanding (Molecular structures)

Installation

To use Docling, simply install docling from your package manager, e.g. pip:

pip install docling

Works on macOS, Linux and Windows environments. Both x86_64 and arm64 architectures.

More detailed installation instructions are available in the docs.

Getting started

To convert individual documents with python, use convert(), for example:

from docling.document_converter import DocumentConverter

source = "https://arxiv.org/pdf/2408.09869"  # document per local path or URL
converter = DocumentConverter()
result = converter.convert(source)
print(result.document.export_to_markdown())  # output: "## Docling Technical Report[...]"

More advanced usage options are available in the docs.

CLI

Docling has a built-in CLI to run conversions.

docling https://arxiv.org/pdf/2206.01062

You can also use 🥚SmolDocling and other VLMs via Docling CLI:

docling --pipeline vlm --vlm-model smoldocling https://arxiv.org/pdf/2206.01062

This will use MLX acceleration on supported Apple Silicon hardware.

Read more here

Documentation

Check out Docling's documentation, for details on installation, usage, concepts, recipes, extensions, and more.

Examples

Go hands-on with our examples, demonstrating how to address different application use cases with Docling.

Integrations

To further accelerate your AI application development, check out Docling's native integrations with popular frameworks and tools.

Get help and support

Please feel free to connect with us using the discussion section.

Technical report

For more details on Docling's inner workings, check out the Docling Technical Report.

Contributing

Please read Contributing to Docling for details.

References

If you use Docling in your projects, please consider citing the following:

@techreport{Docling,
  author = {Deep Search Team},
  month = {8},
  title = {Docling Technical Report},
  url = {https://arxiv.org/abs/2408.09869},
  eprint = {2408.09869},
  doi = {10.48550/arXiv.2408.09869},
  version = {1.0.0},
  year = {2024}
}

License

The Docling codebase is under MIT license. For individual model usage, please refer to the model licenses found in the original packages.

LF AI & Data

Docling is hosted as a project in the LF AI & Data Foundation.

IBM ❤️ Open Source AI

The project was started by the AI for knowledge team at IBM Research Zurich.