watchvideo

English | 中文

Evidence-first video understanding skill for Codex.

watchvideo is a Codex skill plus a local video-analysis CLI. It turns local video files or video URLs into verifiable evidence, then lets Codex summarize from reports, transcripts, OCR, and keyframes. If it cannot produce video evidence, the agent should stop and explain the blocker instead of guessing from the title, description, or search results.

Boundary: the CLI prepares evidence; the agent reads that evidence and writes the final understanding. This project does not call a cloud LLM by itself, and it is not a video editor, transcoder, publisher, or player.

General-purpose agents often take a shortcut when asked to summarize a video: they read the URL, page title, description, or search results and produce a plausible answer. That is useful-looking, but it is not video understanding.

watchvideo closes that shortcut. It requires the workflow to first download or read the video, obtain subtitles or local transcription, extract keyframes, optionally run OCR, and then write evidence packages that an agent can inspect.

Use it when you want to:

• summarize technical talks, interview-prep videos, tutorials, demos, or product walkthroughs;
• keep traceable report.md and summary-input.md files instead of only a chat answer;
• debug Douyin/TikTok/YouTube download failures with explicit diagnostics;
• share a reusable, testable agent skill instead of a one-off prompt.

Install the skill into Codex:

mkdir -p ~/.codex/skills
git clone https://github.com/Dunteng/watch-video-skill ~/.codex/skills/watch-video
cd ~/.codex/skills/watch-video
python3 -m watchvideo doctor

For a standalone CLI installation from the checkout:

python3 -m pip install .
watchvideo --help

Restart your Codex session so $watch-video is discovered, then ask:

Use $watch-video to summarize this video: https://example.com/video

Chinese prompts work too:

用 $watch-video 总结这个视频：https://example.com/video

The CLI prepares evidence. Codex reads the artifacts and writes the final summary. Example outputs:

• examples/technical-interview-summary.en.md
• examples/failure-report.en.md
• examples/download-diagnostics.en.md

Chinese examples are also available under examples/*.zh.md.

The CLI can automatically:

• download video URLs supported by yt-dlp;
• retry yt-dlp with browser cookies when platforms require login or bot verification;
• structurally parse mobile share-page SSR data such as _ROUTER_DATA, RENDER_DATA, and play_addr URLs when yt-dlp fails;
• record every download attempt in the final report;
• download platform subtitles or auto-subtitles;
• probe duration, resolution, and media metadata;
• parse .srt and .vtt subtitles;
• use system whisper, or auto-prepare pinned local whisper.cpp, when subtitles are missing;
• extract keyframes and skip highly similar frames;
• generate report.json, report.md, and summary-input.md;
• check for likely leftover high-load processes.

Optional capabilities:

• run tesseract OCR on keyframes;
• limit the number of keyframes for long videos;
• keep downloaded remote MP4 files with --keep-video.

The CLI does not:

• call OpenAI, Claude, Gemini, or any other cloud LLM;
• edit, cut, compress, transcode, publish, or play videos;
• guarantee that every platform can be downloaded;
• open Chrome UI or operate the browser interface;
• kill processes automatically.

The media-analysis workflow supports macOS and Linux (POSIX). Windows is not currently supported for analyze, doctor, or processes; use WSL or a macOS/Linux host. The pure-Python summarize command remains usable wherever the package can be installed.

Required tools:

python3
yt-dlp
ffmpeg
ffprobe

macOS:

brew install yt-dlp ffmpeg

Optional tools:

git
cmake
whisper
tesseract

git is used when the CLI needs to prepare local whisper.cpp. The default tool cache is ~/Library/Caches/watchvideo on macOS and ${XDG_CACHE_HOME:-~/.cache}/watchvideo on Linux; WATCHVIDEO_TOOLS_DIR or --tools-dir overrides it. If system cmake is missing, the CLI tries a cache-local .venv fallback. Missing optional tools do not block videos that already have usable subtitles, but they may limit transcription or OCR.

Check the environment:

cd ~/.codex/skills/watch-video
python3 -m watchvideo doctor

doctor output:

• REQUIRED_OK: required tool is available;
• REQUIRED_MISSING: install this before analysis;
• OPTIONAL_OK: optional capability is available;
• OPTIONAL_MISSING: workflow can continue, but transcription or OCR may be limited.
• CAPABILITY_READY / CAPABILITY_BLOCKED: whether local-file and remote-URL analysis can run with the detected required tools.

doctor exits non-zero when any required tool is missing. A missing yt-dlp blocks remote URLs, but the local capability can still be ready.

Ask Codex directly:

Use $watch-video to analyze this video: https://example.com/video

or:

Use $watch-video to summarize this local video: /path/to/video.mp4

For summarize/analyze/watch/what-is-this-video-about requests, the agent should run or inspect the CLI artifacts before answering. If report.md exists, the final understanding should be written into the ## 视频内容总结 section unless the user explicitly asks not to write files.

Do not summarize only from a URL, title, description, search result, or same-topic article. The summary must be grounded in MP4 evidence, subtitles/transcripts, OCR, and keyframes. If evidence is missing, report the blocker and ask for a local video file or an accessible direct URL.

If you are not using Codex, run the CLI manually from the skill repository. Save the original workspace first and write outputs back into that workspace:

TASK_WORKDIR="$(pwd)"
cd ~/.codex/skills/watch-video
python3 -m watchvideo analyze "https://example.com/video" \
  -o "$TASK_WORKDIR/analysis/demo"

--max-keyframes defaults to 80 to bound long-video artifacts. Each external command has a two-hour default timeout and bounded captured output; on timeout the whole spawned process group is terminated. Use --command-timeout SECONDS (1-86400) for unusually long inputs. Progress messages identify the current stage without echoing source URLs or prompts.

Local file:

TASK_WORKDIR="$(pwd)"
cd ~/.codex/skills/watch-video
python3 -m watchvideo analyze "$TASK_WORKDIR/video.mp4" \
  -o "$TASK_WORKDIR/analysis/local-video"

Typical higher-signal run:

python3 -m watchvideo analyze "https://example.com/video" \
  --max-height 1080 \
  --keyframe-interval 10 \
  --max-keyframes 80 \
  --sub-lang "zh.*" \
  --sub-lang "en.*" \
  -o "$TASK_WORKDIR/analysis/demo"

Remote MP4 downloads are temporary by default. They are deleted after probe, transcription, keyframe extraction, and OCR finish. report.md and summary-input.md record the cleanup. Keep the downloaded MP4 only when needed:

python3 -m watchvideo analyze "https://example.com/video" \
  --keep-video \
  -o "$TASK_WORKDIR/analysis/demo"

Browser-cookie behavior:

# Disable browser-cookie retry.
python3 -m watchvideo analyze "https://example.com/video" \
  --no-browser-cookies \
  -o "$TASK_WORKDIR/analysis/demo"

# Try chrome, chromium, edge, then firefox.
python3 -m watchvideo analyze "https://example.com/video" \
  --cookies-from-browser auto \
  -o "$TASK_WORKDIR/analysis/demo"

analyze automatically generates the summary input packet. Regenerate it from an existing current-schema report when needed:

python3 -m watchvideo summarize "$TASK_WORKDIR/analysis/demo/report.json" \
  -o "$TASK_WORKDIR/analysis/demo/summary-input.md" \
  --chunk-seconds 300

Check likely leftover processes:

python3 -m watchvideo processes

A successful run normally creates:

• report.json: structured report for programs and follow-up tooling;
• report.md: human-readable report with metadata, evidence quality, timeline preview, download diagnostics, cleanup records, transcription metadata, transcript text, and optional summary;
• summary-input.md: agent-friendly packet with writing requirements, report shape, download diagnostics, cleanup records, transcription metadata, keyframe timestamps, OCR, and chunked transcript;
• current-run.json: current success/failure status and isolated run path;
• .runs/run-*/subtitles/: platform subtitles or auto-subtitles;
• .runs/run-*/transcript/: local transcription output;
• .runs/run-*/keyframes/: extracted keyframe images;
• .runs/run-*/video/: temporary remote download directory; remote MP4s are deleted by default unless --keep-video is used.

analyze and summarize hold the same output-directory lock while reading/publishing state. Success artifacts are staged in that directory and atomically replaced where possible. A successful run removes stale failure.json / failure.md; a failed run removes stale report.json / report.md / summary-input.md, so callers cannot confuse states from different attempts. Old isolated .runs/run-* directories are bounded while the current run is retained.

If no usable evidence can be produced:

• failure.json: structured failure details and download attempts;
• failure.md: human-readable failure report that explicitly forbids title/description/search-based summaries.

Do not commit runtime artifacts. .gitignore excludes analysis/, .tools/, .models/, media files, model files, .env, and cache directories.

Default behavior:

• prefer platform or sidecar subtitles;
• try the system whisper CLI when subtitles are missing;
• if system whisper is unavailable or produces no cues, auto-prepare a pinned whisper.cpp revision in the user cache, verify the base model checksum, and build whisper-cli;
• use WATCHVIDEO_TOOLS_DIR when set; otherwise use ~/Library/Caches/watchvideo on macOS or ${XDG_CACHE_HOME:-~/.cache}/watchvideo on Linux;
• if system cmake is missing, install a pinned compatible cmake into the cache-local .venv fallback;
• never install system-level packages automatically.

report.md records the ASR source, model, language parameter, prompt usage, and transcript files.

Disable automatic whisper.cpp setup:

python3 -m watchvideo analyze "https://example.com/video" \
  --no-auto-transcribe-setup \
  -o "$TASK_WORKDIR/analysis/demo"

Use a custom tools cache:

python3 -m watchvideo analyze "https://example.com/video" \
  --tools-dir "$TASK_WORKDIR/.tools" \
  --whisper-prompt "Agent, Codex, AI coding" \
  --language zh \
  -o "$TASK_WORKDIR/analysis/demo"

Use preinstalled whisper.cpp:

python3 -m watchvideo analyze "https://example.com/video" \
  --whisper-cpp-bin .tools/whisper.cpp/build/bin/whisper-cli \
  --whisper-model .tools/whisper.cpp/models/ggml-base.bin \
  --whisper-prompt "Agent, Codex, AI coding" \
  --language zh \
  --max-keyframes 80 \
  -o "$TASK_WORKDIR/analysis/demo"

Enable OCR:

python3 -m watchvideo analyze "$TASK_WORKDIR/video.mp4" \
  --ocr \
  -o "$TASK_WORKDIR/analysis/local-video"

If tesseract is missing, the main workflow continues and records a warning.

The project uses stdlib unittest; pytest is not required.

python3 -m unittest discover -s tests -v
python3 -m compileall watchvideo tests
python3 -m pip wheel --no-deps .
python3 -m watchvideo doctor

The downloader tests include sanitized share-page fixtures for _ROUTER_DATA, play_addr.url_list, aweme/v1/playwm, and CDN MP4 redirects.

GitHub Actions runs the full test suite, compileall, wheel build, isolated installation, and watchvideo --help smoke test on Python 3.11-3.14.

doctor must not show REQUIRED_MISSING. OPTIONAL_MISSING is acceptable, but it affects transcription or OCR capabilities.

• README.zh-CN.md: Chinese README.
• docs/ARCHITECTURE.md: module boundaries and data flow.
• docs/OPERATIONS.md: local operations, transcription, OCR, and process checks.
• docs/PUBLISHING.md: public repository checklist.
• docs/PROMOTION.md: public positioning, launch copy, and demo script.
• examples/: sanitized example reports and download diagnostics.
• evals/skill_scenarios.md: agent behavior scenarios.

Keep these directories and files in the long-term reusable skill:

• SKILL.md
• references/
• watchvideo/
• scripts/
• tests/
• evals/
• docs/
• examples/

Before publishing, confirm that the repository does not include real videos, audio, subtitles, keyframes, transcripts, cookies, tokens, .env, private links, or user Obsidian notes.