Many PDF files are long, and sometimes goes beyond 100 pages which is the maximum page limit for the Document Parse API.
To process such long documents, or even to get faster responses for easier debugging, we can split our long PDF to shorter PDFs.
Given a PDF filepaper.pdf,
run the code below to split a PDF into shorter PDFs and save them to separate files.
Note that we saved the example PDF file above and named it to paper.pdf.
If you want to split, send, and merge results in a single script without saving intermediate files, see Synchronous API with chunking.
"""Requirements: `pip install pymupdf` to import fitz"""import osimport fitzdef split_pdf(input_file, batch_size): # Open input_pdf input_pdf = fitz.open(input_file) num_pages = len(input_pdf) print(f"Total number of pages: {num_pages}") # Split input_pdf for start_page in range(0, num_pages, batch_size): end_page = min(start_page + batch_size, num_pages) - 1 # Write output_pdf to file input_file_basename = os.path.splitext(input_file)[0] output_file = f"{input_file_basename}_{start_page}_{end_page}.pdf" print(output_file) with fitz.open() as output_pdf: output_pdf.insert_pdf(input_pdf, from_page=start_page, to_page=end_page) output_pdf.save(output_file) # Close input_pdf input_pdf.close()# Input argumentsinput_file = "paper.pdf" # Replace with a file of your ownbatch_size = 10 # Maximum available value is 100split_pdf(input_file, batch_size)
You can see resulting files in the terminal as follows.
$ ls paper*paper.pdf paper_0_9.pdf paper_10_12.pdf
Document Parse offers base64 encoding for layout elements that users wish to extract as images. Let's attempt to extract all figures and save them as image files. You can add {'base64_encoding': '["figure"]'} to the request body, and the API will return a base64 encoded string for all figure elements. The following code then parses the response and saves the first figure to an image file.
"""Requirements: `pip install requests`"""from glob import globimport jsonimport osimport requestsAPI_KEY = "UPSTAGE_API_KEY"def call_document_parse(input_file, output_file): # Send request response = requests.post( "https://api.upstage.ai/v1/document-digitization", headers={"Authorization": f"Bearer {API_KEY}"}, data={"base64_encoding": "['figure']", "model": "document-parse"}, # base64 encoding for cropped image of the figure category. files={"document": open(input_file, "rb")}) # Save response if response.status_code == 200: with open(output_file, "w") as f: json.dump(response.json(), f, ensure_ascii=False) else: raise ValueError(f"Unexpected status code {response.status_code}.")# Find all shorter PDFs related to input_fileinput_file = "paper.pdf"short_input_files = glob(os.path.splitext(input_file)[0] + "_*.pdf")# Send request and save response for all shorter PDFsfor short_input_file in short_input_files: print(short_input_file) short_output_file = os.path.splitext(short_input_file)[0] + ".json" call_document_parse(short_input_file, short_output_file)
import jsonimport base64# Input parametersjson_file = "paper_0_9.json"output_file = "paper_cropped_figure.png"# Load JSON file to get bounding box of the first figurewith open(json_file, "r") as f: data = json.load(f) # Get bounding box for the first figure and crop the image for element in data["elements"]: if element["category"] == "figure": with open (output_file, 'wb') as fh: fh.write(base64.decodebytes(str.encode(element["base64_encoding"]))) break
The complete flow for cropping a figure and saving it as an image file is as follows.
Cropping a figure in 3 steps: source recognition β base64 response β image file save
