How to Run an AI OCR Document Reader Agent with OE Runtime

Extract text from scanned documents, invoices, receipts, and forms automatically. OE Runtime uses Azure Vision to run OCR, classifies the document type, and structures the extracted data into ready-to-use fields — no manual data entry required.

What You Need

• Node.js 18+ installed, or the OE Runtime binary from openenthrium.com/runtime
• An API key from OpenAI, Anthropic, or any supported LLM provider
• An Azure Computer Vision API key for OCR
• A text editor — nothing else

Create the Project Folder

Create a folder called ocr-vision/ and add these two files:

The Skill Files

Create SKILL.md inside an ocr-vision/ directory. This is the portable skill — it follows the agentskills.io format and runs unchanged on Claude, Cursor, Windsurf, or OE Runtime:

---
name: ocr-vision
description: Extract and analyze text from images using OCR
license: Apache-2.0
metadata:
  author: Open Enthrium
  version: "1.0"
---

You are a document analysis agent. Extract text from images using OCR,
then analyze, classify, and summarize the extracted content.
Complete all steps fully before writing your report.

## Step 1: Extract Text
Call the Azure Vision connector to run OCR on the provided image.
POST /read/analyze with body:
{
  "url": "<image URL to analyze>"
}
Save the Operation-Location URL from the response headers.
Then poll GET <operation-path> (the path portion of the Operation-Location URL)
every 2 seconds until status is "succeeded".
Extract all text lines from the analyzeResult.readResults.

## Step 2: Classify and Structure
Analyze the extracted text to determine the document type
(invoice, receipt, contract, ID, form, letter, or other).
Extract key structured fields based on type — e.g. for an invoice: vendor, date, line items, total.

## Step 3: Report
Produce a document analysis result:
- Document type identified
- Structured fields extracted with values
- Full raw text extracted
- Any low-confidence regions or unreadable sections
- Suggested next action (e.g. file, process payment, follow up)

Create agent.yaml in the same directory to wire the skill to your connector:

name: Document Reader
description: Extract and analyze text from images using OCR
connectors:
  - connection_name: Azure Vision
    connection_type: azure-vision
skills:
  - path: ./
    trigger_type: auto

The Config File

Create oe-config.json in the same directory:

{
  "llm": {
    "provider": "openai",
    "model": "gpt-4o",
    "apiKey": "YOUR_OPENAI_API_KEY"
  },
  "server": {
    "enabled": false,
    "port": 3333,
    "apiKey": "your-secret-api-key"
  },
  "connectors": [
    {
      "connection_name": "Azure Vision",
      "connection_type": "azure-vision",
      "apiKey": "YOUR_AZURE_VISION_KEY",
      "headerName": "Ocp-Apim-Subscription-Key",
      "baseUrl": "https://YOUR_RESOURCE.cognitiveservices.azure.com/vision/v3.2"
    }
  ]
}

Create an Azure Computer Vision resource at portal.azure.com and copy the endpoint and API key from the resource's Keys and Endpoint page. The free tier (F0) allows up to 5,000 OCR transactions per month.

Download OE Runtime

Run the Agent

From the parent folder containing your skill directory:

1 npx recommended

npx -y @openenthrium/oe-runtime@latest ./ocr-vision

2 Windows

oe-runtime-win.exe ./ocr-vision

3 macOS

chmod +x oe-runtime-macos
./oe-runtime-macos ./ocr-vision

First run blocked? System Settings → Privacy & Security → Allow Anyway.

4 Linux

chmod +x oe-runtime-linux
./oe-runtime-linux ./ocr-vision

5 API Server integration

Add a "server" block to oe-config.json, then start with --serve:

{
  "llm": { ... },
  "server": { "enabled": true, "port": 3333, "apiKey": "your-secret-key" },
  "connectors": [ ... ]
}

npx -y @openenthrium/oe-runtime@latest --serve --config oe-config.json

Run with inline YAML:

curl -X POST http://localhost:3333/run \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-secret-key" \
  -d '{"yaml": "...", "params": {}}'

Or run from a file on the server:

curl -X POST http://localhost:3333/run-file \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-secret-key" \
  -d '{"file": "/path/to/agent.yaml", "params": {}}'

Use Cases

• Invoice processing automation — Extract vendor name, invoice number, line items, amounts, and due dates from scanned or photographed invoices automatically. The structured output can be fed directly into your accounts payable system, eliminating manual data entry and reducing processing time from days to seconds.
• Receipt digitisation — Convert paper receipts into structured expense records by photographing them and running the OCR agent. The agent extracts merchant name, date, amount, and category, ready to import into your expense management system or spreadsheet.
• Contract data extraction — Parse legal contracts and agreements to extract key clauses, parties, dates, payment terms, and renewal conditions. The agent surfaces the critical information from long documents so your legal or procurement team can review what matters without reading every page.
• ID and form verification — Extract data from identity documents, application forms, and registration papers to populate database records automatically. This reduces manual transcription errors in KYC, onboarding, and registration workflows where accuracy is critical.
• Medical record processing — Extract structured data from handwritten or printed medical forms, referral letters, and lab reports. The agent can identify patient details, diagnosis codes, medication names, and test results to feed into electronic health record systems.
• Logistics and shipping document processing — Process bills of lading, packing lists, and customs declaration forms to extract shipment details, quantities, and consignee information. Automating this step in import/export workflows significantly reduces manual processing time and customs clearance delays.
• Historical document digitisation — Convert archives of scanned historical documents, letters, or records into searchable digital text. The agent processes each document, extracts the readable content, and flags sections with low OCR confidence for manual review.
• Quality control and compliance — Read product labels, certificates of conformity, and inspection reports from photographed documents to verify that required fields and values are present. The agent can flag documents where mandatory information is missing or illegible before they are filed.