Document classification

What is Document classification?

Document classification classifies an input document into one of your user-defined categories. It works across scanned images, photos, and born-digital PDFs, and requires no templates or fine-tuning.

Document classification is a powerful tool for automating document processing pipelines. For example, if user have bunch of documents and several extraction tasks, user can use document classification to classify the document into different types. Then, user can use Information Extract by mapping the document type to the appropriate schema for the extraction task.

Why use Upstage Document classification?

• Any document: Robust across scans, photos, and digital documents
• Flexible taxonomy: Freely define and update labels to fit your domain
• No fine-tuning required: Deploy immediately without templates/training, lowering ops cost
• Consistent operations: Use directly for document routing and automation branching

What models does Upstage provide?

Sending a request in three simple steps

{
  "type": "image_url",
  "image_url": {"url": f"data:application/octet-stream;base64,{base64_data}"}
}

{
  "type": "json_schema",
  "json_schema": {
    "name": "document-classify",
    "schema": {
      "type": "string",
      "oneOf": [
        {"const": "invoice", "description": "Commercial invoice with itemized charges and billing information"},
        {"const": "receipt", "description": "Receipt showing purchase transaction details"},
        {"const": "contract", "description": "Legal agreement or contract document"},
        {"const": "cv", "description": "Curriculum vitae or resume"},
        {"const": "others", "description": "Other"}
      ]
    }
  }
}

result = response.choices[0].message.content
print(result)

"invoice"

Writing a class description

Follow these recommendations to design labels and descriptions for better accuracy:

• Clear label names: Use concrete, non-overlapping names (e.g., invoice, receipt, contract).
• Differentiating descriptions: State key criteria in 1–2 sentences and add decision rules for borderline cases.
• Use examples (optional): Brief include/exclude examples help with ambiguous cases.
• Manage label count: Too many labels can confuse the model. Start with core labels and expand gradually.
• Consistent format: Keep label casing and separators (lowercase, hyphen/underscore) consistent.

Example:

{
  "type": "json_schema",
  "json_schema": {
    "name": "document-classify",
    "schema": {
      "type": "string",
      "oneOf": [
        {"const": "invoice", "description": "Business invoice with seller/buyer info and itemized amounts"},
        {"const": "receipt", "description": "Retail receipt with payment method/change and simple item list"},
        {"const": "contract", "description": "Legal document with parties, clauses, and signatures/stamps"},
        {"const": "cv", "description": "Curriculum vitae or resume summarizing work history and skills"},
        {"const": "others", "description": "Documents not covered by the above labels"}
      ]
    }
  }
}

Class design restrictions

• The number of classes may have up to 1000 values
• The number of classes must be at least 1.
• Classes with duplicate names are ignored.

Getting the score of the class

The model returns the class with the highest score. The score is a value between 0 and 1, where 1 is the highest score.

Users can get the score of the class by using the additional_values tool call.

score_dict = response.choices[0].message.tool_calls[0].function.arguments
print(score_dict)

{"document_type": {"_value": "invoice", "confidence_score": 1.0}}

Examples

Request

# pip install openai
import base64
import json
from openai import OpenAI

client = OpenAI(
    api_key="UPSTAGE_API_KEY",
    base_url="https://api.upstage.ai/v1/document-classification"
)

def encode_img_to_base64(img_path):
    with open(img_path, 'rb') as img_file:
        img_bytes = img_file.read()
        base64_data = base64.b64encode(img_bytes).decode('utf-8')
        return base64_data

filepath = "./your_document.png"
base64_data = encode_img_to_base64(filepath)

response = client.chat.completions.create(
    model="document-classify",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:application/octet-stream;base64,{base64_data}"}
                }
            ]
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "document-classify",
            "schema": {
                "type": "string",
                "oneOf": [
                    {"const": "invoice", "description": "Commercial invoice with itemized charges and billing information"},
                    {"const": "receipt", "description": "Receipt showing purchase transaction details"},
                    {"const": "contract", "description": "Legal agreement or contract document"},
                    {"const": "cv", "description": "Curriculum vitae or resume"},
                    {"const": "others", "description": "Other"}
                ]
            }
        }
    }
)

print(response) 

Response

{
  "id": "8f111e4201de097efa38d4a4d174c001",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": None,
      "message": {
        "content": "invoice",
        "refusal": None,
        "role": "assistant",
        "annotations": None,
        "audio": None,
        "function_call": None,
        "tool_calls": [{
          "id": None,
          "function": {
            "arguments": {"document_type": {"_value": "invoice", "confidence_score": 1.0}},
            "name": "additional_values"
          },
          "type": "function"
        }]
      }
    }
  ],
  "created": 1756270462,
  "model": "document-classify-260904",
  "object": None,
  "service_tier": None,
  "system_fingerprint": None,
  "usage": {
    "completion_tokens": 6,
    "prompt_tokens": 1765,
    "total_tokens": 1771,
    "completion_tokens_details": None,
    "prompt_tokens_details": None
  }
}