Skip to main content

HuridocsPDFLoader

Segments the PDF and returns a list of Document items where every item is a segment.
These segments contain detailed metadata about the PDF and its pages.

This is an implementation of self-hosted pdf-document-layout-analysis service.

Overview​

Integration details​

ClassPackageLocalSerializableJS support
HuridocsPDFLoaderlangchain_community✅❌❌

Loader features​

SourceDocument Lazy LoadingNative Async Support
HuridocsPDFLoader✅❌

Setup​

Credentials​

No credentials are needed to use this loader.

If you want to get automated best in-class tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
# os.environ["LANGSMITH_TRACING"] = "true"

Installation​

Install langchain_community.

%pip install -qU langchain_community

To install and run the layout analysis service:

!git clone http://github.com/huridocs/pdf-document-layout-analysis.git
%cd pdf-document-layout-analysis
!make start

Initialization​

Now we can instantiate our model object and load documents:

from langchain_community.document_loaders import HuridocsPDFLoader
from langchain_core.documents import Document

loader = HuridocsPDFLoader("./example_data/layout-parser-paper.pdf")
API Reference:HuridocsPDFLoader | Document

Load​

docs: list[Document] = loader.load()
docs[0]
Document(metadata={'coordinates': (158.0, 115.0, 298.0, 30.0), 'page_number': 1, 'page_width': 612, 'page_height': 792, 'type': 'Title'}, page_content='LayoutParser : A Unified Toolkit for Deep Learning Based Document Image Analysis')
docs[1]
Document(metadata={'coordinates': (134.0, 170.0, 346.0, 22.0), 'page_number': 1, 'page_width': 612, 'page_height': 792, 'type': 'Text'}, page_content='Zejiang Shen 1 ( ), Ruochen Zhang 2 , Melissa Dell 3 , Benjamin Charles Germain Lee 4 , Jacob Carlson 3 , and Weining Li 5')
docs[2]
Document(metadata={'coordinates': (207.0, 203.0, 199.0, 107.0), 'page_number': 1, 'page_width': 612, 'page_height': 792, 'type': 'Text'}, page_content='1 Allen Institute for AI shannons@allenai.org 2 Brown University ruochen zhang@brown.edu 3 Harvard University { melissadell,jacob carlson } @fas.harvard.edu 4 University of Washington bcgl@cs.washington.edu 5 University of Waterloo w422li@uwaterloo.ca')
print(docs[0].metadata)
{'coordinates': (158.0, 115.0, 298.0, 30.0), 'page_number': 1, 'page_width': 612, 'page_height': 792, 'type': 'Title'}

Lazy Load​

page = []
for doc in loader.lazy_load():
    page.append(doc)
    if len(page) >= 10:
        # do some paged operation, e.g.
        # index.upsert(page)

        page = []

API reference​

For detailed documentation of all PDFMinerLoader features and configurations head to the API reference: https://api.python.langchain.com/en/latest/document_loaders/langchain_community.document_loaders.pdf.HuridocsPDFLoader.html

Was this page helpful?


You can also leave detailed feedback on GitHub.