Every VPP partner, ad-tech vendor, and post house is building roughly the same scene-analysis layer in-house. Segmentation, tracking, OCR, face ID, shot chunking. It’s duplicated R&D, and most of those teams stall before they ship anything ad sales can use.
The Scene Intelligence Engine is the API layer underneath. Frame-by-frame object taxonomy, temporal-spatial metadata, and replacement-ready exports in four output formats. The signal that moves CPMs on contextual inventory and unlocks scene-level direct-sold packages.
Built on SAM3, YOLO26, and DiffuEraser, with an ASR layer tuned for Korean, Japanese, and Mandarin dialogue. Designed for catalog work, not for one clip at a time.
Inputs in, exports out. Every stage is versioned and scaled independently and emits its own evaluation telemetry. You can fail soft, re-run, or swap any stage without touching the others.
Chunk source media. Extract video, audio tracks, embedded subs. Normalize timecode to broadcast standard (SMPTE).
SAM3 + YOLO26 across every frame. Object masks, classes, persistence IDs. Whisper + Clova on dialogue. OCR on text.
Persist objects across cuts. Identify products, faces, brands, locations. Build the temporal-spatial knowledge graph.
LLM layer adds scene captions, mood, beat structure, and brand-safety flags. Outputs IAB taxonomy + custom ontologies.
Rights-aware exports: JSON sidecar, CMAF metadata, replacement-ready assets, C2PA-signed outputs.
Identify branded surfaces (products, packaging, signage) and prepare replacement anchors. Plug into VPP partners or build your own pipeline. Opens scene-level direct-sold inventory that doesn’t interrupt playback, and complements rather than competes with AVOD pods.
Mood, scene, brand-safety, and IAB-v3 signals exported per shot and per minute. Plugs into Samba, GumGum, IRIS.TV, or your own SSP. Lifts CPMs on premium inventory and opens fill from advertisers who won’t buy uncertified context.
Multimodal search across long-form catalogs (“night exterior, two speaking characters, a vehicle”), plus scene-essence embeddings for “more like this” rails. Drives session length and SVOD retention without forcing a recsys re-platform.
Shot chunking with scene grammar, action beats, and dialogue alignment, delivered as FCPXML or Resolve-compatible markers. Editors keep the cut. The AI does the timecodes, the scene logs, and the dialogue spotting.
EU AI Act Article 50 watermarking, C2PA-signed exports, and rights metadata at frame granularity. Treat AI-touched media as the regulated artifact it’s becoming, and keep deliveries on schedule when audits land.
Submit a video URL. Get back a JSON sidecar with every object on every frame. Stream events as they’re detected, take a webhook on completion, or mux CMAF metadata directly into your HLS or DASH manifest.
Per-tenant auth. Rate-limited, observable, idempotent. Usage-based pricing tied to source-hours processed, so the bill scales with the catalog rather than with seats.
# SUBMIT JOB curl -X POST https://api.kladon.ai/v1/scenes \ -H "Authorization: Bearer $KEY" \ -d '{ "source": "s3://catalog/ep-014.mp4", "lang": "ko", "exports": ["json", "cmaf", "c2pa"] }' # RESPONSE { "job_id": "k-7c4e2a", "status": "queued", "eta_sec": 412, "webhook": "https://you/hook" }
You bring the catalog.
We bring the frames-to-pixels stack.