How to Run an AI Video Generation Agent with OE Runtime

Generate short cinematic video clips from text descriptions automatically. OE Runtime connects to Runway, crafts detailed scene prompts, submits the generation job, polls for completion, and returns the video URL — no video production skills required.

What You Need

• Node.js 18+ installed, or the OE Runtime binary from openenthrium.com/runtime
• An API key from OpenAI, Anthropic, or any supported LLM provider
• A Runway API key for AI video generation
• A text editor — nothing else

Create the Project Folder

Create a folder called video-generation/ and add these two files:

The Skill Files

Create SKILL.md inside a video-generation/ directory. This is the portable skill — it follows the agentskills.io format and runs unchanged on Claude, Cursor, Windsurf, or OE Runtime:

---
name: video-generation
description: Generate short video clips from text prompts
license: Apache-2.0
metadata:
  author: Open Enthrium
  version: "1.0"
---

You are a video generation agent. Create short video clips using Runway.
Video generation is asynchronous — after submitting, poll for completion before returning the result.
Complete all steps fully before writing your report.

## Step 1: Design Video Prompt
Craft a detailed Runway video prompt for a 5-second product reveal clip.
Scene: a sleek smartphone emerging from mist on a dark reflective surface.
Motion: slow upward drift with a rotating reveal.
Style: cinematic, high contrast, blue accent lighting.
Camera: slow push-in, shallow depth of field.

## Step 2: Generate Video
Submit the prompt to the Runway connector.
POST /image_to_video with body:
{
  "promptText": "<video prompt from previous step>",
  "model": "gen3a_turbo",
  "duration": 5,
  "ratio": "1280:720"
}
Save the id from the response.
Poll GET /tasks/<id> every 15 seconds until status is "SUCCEEDED".
Once complete, extract and return the output video URL.

## Step 3: Report
Summarize the result:
- Prompt used for generation
- Generation time taken
- Output video URL
- Duration and resolution

Create agent.yaml in the same directory to wire the skill to your connector:

name: Video Creator
description: Generate short video clips from text prompts
connectors:
  - connection_name: Runway
    connection_type: runway
skills:
  - path: ./
    trigger_type: auto

The Config File

Create oe-config.json in the same directory:

{
  "llm": {
    "provider": "openai",
    "model": "gpt-4o",
    "apiKey": "YOUR_OPENAI_API_KEY"
  },
  "server": {
    "enabled": false,
    "port": 3333,
    "apiKey": "your-secret-api-key"
  },
  "connectors": [
    {
      "connection_name": "Runway",
      "connection_type": "runway",
      "bearerToken": "YOUR_RUNWAY_API_KEY",
      "baseUrl": "https://api.dev.runwayml.com/v1"
    }
  ]
}

Get your Runway API key from app.runwayml.com/settings. The model gen3a_turbo is the fastest generation option. The duration field accepts 5 or 10 seconds. Video generation is asynchronous — the agent polls until the job completes before returning the URL.

Download OE Runtime

Run the Agent

From the parent folder containing your skill directory:

1 npx recommended

npx -y @openenthrium/oe-runtime@latest ./video-generation

2 Windows

oe-runtime-win.exe ./video-generation

3 macOS

chmod +x oe-runtime-macos
./oe-runtime-macos ./video-generation

First run blocked? System Settings → Privacy & Security → Allow Anyway.

4 Linux

chmod +x oe-runtime-linux
./oe-runtime-linux ./video-generation

5 API Server integration

Add a "server" block to oe-config.json, then start with --serve:

{
  "llm": { ... },
  "server": { "enabled": true, "port": 3333, "apiKey": "your-secret-key" },
  "connectors": [ ... ]
}

npx -y @openenthrium/oe-runtime@latest --serve --config oe-config.json

Run with inline YAML:

curl -X POST http://localhost:3333/run \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-secret-key" \
  -d '{"yaml": "...", "params": {}}'

Or run from a file on the server:

curl -X POST http://localhost:3333/run-file \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-secret-key" \
  -d '{"file": "/path/to/agent.yaml", "params": {}}'

Use Cases

• Product reveal and teaser videos — Generate cinematic product reveal clips from text descriptions for use in social media, email campaigns, and landing pages. The agent crafts detailed prompts with lighting, motion, and camera direction to produce professional-looking clips without a production budget.
• Social media content creation — Produce short video content for Instagram Reels, TikTok, LinkedIn, and Twitter at scale without a video production team. Describe the scene, mood, and style and the agent generates a unique clip ready for posting in minutes rather than days.
• Marketing campaign visuals — Create compelling background video loops for landing pages, digital ads, and video banners that align with your campaign's visual direction. Swapping visual concepts is as simple as changing the text prompt, enabling rapid creative iteration.
• Explainer and concept visualisation — Turn abstract product concepts, architectural ideas, or technology descriptions into visual clips that communicate what words alone cannot. The agent's ability to render motion and atmosphere makes it ideal for early-stage concept communication before physical assets exist.
• E-learning and training content — Generate illustrative video clips to accompany instructional content, making abstract concepts more engaging and easier to understand. Video components increase learner engagement and knowledge retention compared to text and static images alone.
• Real estate and architectural visualisation — Produce cinematic exterior or interior video walkthroughs of properties or architectural concepts from descriptive prompts. These clips can be used in property listings, investor pitches, and planning presentations before construction begins.
• Game and app trailers — Generate stylised video clips showcasing game environments, characters, or app interfaces for use in store listings, trailers, and promotional material. Multiple visual styles can be explored quickly by iterating on the prompt without engaging a production studio.
• News and editorial illustration — Produce illustrative video segments to accompany news articles, blog posts, or editorial content that lack original visual assets. Publications can generate contextually appropriate video to accompany written stories without sourcing stock footage.