# svc-media-toolkit — instructions for AI agents

Internal Alterspective media MCP service. Audio/video transcription and a
video-improvement pipeline (silence cleanup, loudness normalisation, branded
intro/outro cards with a burned-in classification banner). Self-hosted — no
client media leaves to a public cloud STT.

MCP endpoint: `https://svc-media-toolkit.alterspective.com.au/mcp` (Streamable HTTP, protocol `2025-06-18`)
Full technical reference (architecture, known limitations): KB-AI-MCP-001 in
the Alterspective-Intelligence KB.

## Authentication

Every `/mcp` request requires `Authorization: Bearer <token>` where
`<token>` is a Keystone API key carrying the `mcp:media` scope. The gate
fails closed: no token → `401`; wrong scope → `403`; misconfigured server →
`503` (not your fault, a deploy issue).

To get a token: use the `keystone` MCP's `create-api-key` tool with
`ownerType: "service"` and `scopes: ["mcp:media"]`.

## Important constraint: no file upload

**Every tool below takes a filesystem path, not file bytes.** This service
and the caller must share a filesystem (same host, or a mounted volume) — there
is currently no way to upload a file over MCP. If you can't already see the
target file on this service's filesystem, you cannot use these tools on it yet.

## Tools

All write tools are **submit/poll**: the call returns `{ jobId }`
immediately; poll with `job_status`/`job_result` until the status is
`succeeded` or `failed`. Jobs run for minutes (transcode, transcription) —
don't expect an inline result.

### `media_transcribe`
Audio/video → markdown transcript.

```json
{ "file": "/absolute/path/to/media.mp4", "speakers": 2, "language": "en", "allowCloudFallback": false }
```
- `file` (required) — absolute path, must resolve inside the server's allowed roots.
- `speakers` (optional) — hint for diarization. **Currently has no effect on any
  backend** — none of the three routes below perform diarization yet.
- `language` (optional) — ISO code or `"auto"`.
- `allowCloudFallback` (optional, default `false`) — **residency-breaking, opt-in
  only.** Synapse does not currently support audio transcription (confirmed:
  `GET /v1/models` lists `openai/whisper-1` as `supported: false`), so without this
  flag the only backend is the bundled CPU faster-whisper CLI, which is slow for large
  files. Setting this to `true` routes the extracted audio to a **cloud** model
  (`gemini/gemini-2.5-flash`, pinned via Synapse) instead of using local
  CPU. **Never set this for client media** — audio leaves the on-prem boundary.
- Result: `{ markdown, backend }` — `backend` is one of: `"synapse"` (if
  `MEDIA_STT_BASE_URL` is ever configured against a working Synapse STT endpoint —
  not currently the case in production), `"cloud"` (if `allowCloudFallback:true` was
  set), or `"local-cli"` (the default). Don't assume which one is live — check the
  returned `backend` field.

### `media_cleanup`
Remove silence/dead-air from a recording (auto-editor).

```json
{ "file": "/path/in.mp4", "out": "/path/out.mp4", "threshold": 0.04, "marginSec": 0.2 }
```
Result: `{ out }`.

### `media_enhance_audio`
Loudness-normalise to EBU R128 (-16 LUFS).

```json
{ "file": "/path/in.mp4", "out": "/path/out.mp4" }
```
Result: `{ out }`.

### `media_bookend`
Branded intro/outro cards with a classification banner, optionally burned as a
persistent watermark across the whole video.

```json
{
  "sourceVideo": "/path/in.mp4",
  "outputVideo": "/path/out.mp4",
  "classification": { "level": "CONFIDENTIAL", "audience": "SLT", "watermark": true },
  "intro": { "eyebrow": "Internal Demo", "titleLine1": "My", "titleLine2": "Video", "subtitle": "..." },
  "outro": { "eyebrow": "In Short", "titleLine1": "Thanks", "titleLine2": "for watching", "subtitle": "..." },
  "valuesLine": "EXPERT · APPROACHABLE · TRUSTWORTHY · FORWARD-THINKING",
  "introSeconds": 4,
  "outroSeconds": 5,
  "brand": {
    "assetsDir": "/app/brand-assets",
    "reversedLogo": "logo_reversed.png",
    "chronicleSemibold": "ChronicleDisplay-Semibold.otf",
    "montserratRegular": "Montserrat-Regular.ttf",
    "montserratSemibold": "Montserrat-SemiBold.ttf"
  }
}
```
- `classification.level` — one of `PUBLIC` | `INTERNAL` | `CONFIDENTIAL` | `RESTRICTED`.
  `doNotDistribute` and `watermark` default to `true` for CONFIDENTIAL/RESTRICTED, `false` otherwise.
- `brand` — **required on every call** (the schema has no default). The
  `assetsDir` above (`/app/brand-assets`) is THIS deployment's actual configured
  brand directory — pass it verbatim unless you need different branding.
- Result: `{ outputVideo, watermarked, classification }`.

### `media_suggest_bookend_copy`
Suggest `media_bookend` intro/outro copy (eyebrow/title lines/subtitle) and a
`valuesLine` from a transcript. **Synchronous** — not submit/poll, returns in
seconds. Routes through the Synapse AI gateway (KB-AI-021).

```json
{ "transcript": "# Transcript\n\n...", "residencySafe": true }
```
- `transcript` (required) — e.g. the `markdown` field from a `media_transcribe` result.
- `residencySafe` (optional, default `true`) — pins the request to a local-only
  model (`x-privacy-tier: local-only`) even under auto-routing. Set `false` only
  for non-client content where cloud-quality output is worth the trade — never
  for client media.
- Result: `{ intro: {eyebrow, titleLine1, titleLine2, subtitle}, outro: {...}, valuesLine }`
  — feed this straight into `media_bookend`'s `intro`/`outro`/`valuesLine` fields.

### `job_status`
`{ "jobId": "..." }` → `{ id, kind, status, progress, error, createdAt, updatedAt }`.

### `job_result`
`{ "jobId": "..." }` → `{ status: "succeeded", result }` or an `isError` response
for `"failed"`/not-found. A not-yet-finished job returns its status/progress,
not an error.

## Worked example (submit → poll)

1. Call `media_bookend` with the payload above → get back `{ jobId: "abc-123" }`.
2. Call `job_status` with `{ jobId: "abc-123" }` every few seconds until
   `status` is `"succeeded"` or `"failed"`.
3. Call `job_result` with the same `jobId` to get the final output path.

## Known limitations (be upfront about these, don't guess around them)

- No speaker diarization on the live transcription path (see `media_transcribe` above).
- No file-upload primitive (see the constraint above) — same-filesystem callers only.
- No visual (frame-level) video understanding, transcript translation, or YouTube ingest yet — documented routing points in the code, not implemented.

If a task needs any of the above, say so explicitly rather than attempting a
workaround that silently produces a worse result (e.g. claiming a transcript
is diarized when it isn't).