Status: implementation-ready process spec
Owner: local coding agent
Reference run: workshop-followup/
Primary output: verified follow-up materials from one or more source videos
This process turns a raw workshop recording, tutorial, app walkthrough, bug reproduction, or short screen recording into a reviewable artifact project. It is built from the completed workshop-followup/ run and the Oracle review session video-pipeline-repeatable-process.
The pipeline has two main paths:
The core rule is evidence preservation. Keep the original media immutable, preserve original timestamps when a source timeline exists, save model prompts/results/logs, and keep verification receipts next to every generated claim.
Operational rule from the 2026-06 YouTube batch runs: Gemini video analysis must use
Oracle browser mode and must clean up after itself. After a chunk's analysis is saved
and verified, close the Gemini tab opened for that chunk. Leave the browser/profile
alive for reuse, but do not leave accumulated gemini.google.com/app tabs on the
remote Mac mini.
Use one run folder per video or video set:
runs/
YYYY-MM-DD-slug/
README.md
run-config.json
inputs/
source-video.mp4
source-video.sha256
transcript.vtt
transcript.md
chat.txt
chat.md
screenshots/
clips/
receipts/
ffprobe.source.json
media-info.md
source-file-list.txt
tool-versions.txt
git-status.before.txt
git-status.after.txt
analysis/
timeline-initial.json
silencedetect.log
boundary-candidates.md
segmentation-notes.md
source-timeline.md
segments/
segment-manifest.v1.json
segment-manifest.v1.md
segment-manifest.v2.json
segment-manifest.v2.md
break-chat/
morning-break.chat.md
lunch-break.chat.md
other-break.chat.md
chunks/
video-manifest.json
01/
chunk.mp4
chunk.ffprobe.json
transcript.md
chat.md
carry-forward-chat.md
next-chunk-preview.md
gemini-prompt.md
expected-sentinel.txt
gemini-run.log
gemini-result.json
gemini-analysis.md
gemini-analysis.extracted.json
gemini-omissions-prompt.md
gemini-omissions-run.log
gemini-omissions-result.json
gemini-omissions.md
final-reconciled.md
final-reconciled.json
keyframes/
candidates.json
extracted/
strips/
contact-sheet.jpg
review.md
verification/
expected-items.json
extracted-items.json
coverage-report.md
hallucination-report.md
timestamp-report.md
qa-report.md
visual-claim-report.md
oracle/
planning.md
review.md
sessions/
final/
README.md
student-followup-guide.md
chapter-index.md
resources-and-links.md
qa-index.md
exercises.md
keyframes-manifest.md
keyframes-manifest.json
instructor-review-checklist.md
publication/
public-draft/
private-redactions.md
publish-checklist.md
The current reference artifacts live in workshop-followup/. New runs should use runs/<date>-<slug>/ so the reference run remains untouched.
Create run-config.json before processing:
{
"run_id": "2026-05-30-example-workshop",
"title": "Example Workshop",
"mode": "long_workshop",
"source": {
"video": "inputs/source-video.mp4",
"transcript": "inputs/transcript.vtt",
"chat": "inputs/chat.txt",
"screenshots": "inputs/screenshots",
"clips": "inputs/clips"
},
"policy": {
"preserve_original_timestamps": true,
"canonical_chapters": "semantic",
"canonical_chapter_max_minutes": 30,
"analysis_shard_target_minutes": 4,
"analysis_shard_max_minutes": 4,
"analysis_attachment_target_mib": 9,
"analysis_media": "h264-main-yuv420p-29.97fps-aac-lc-44.1khz-faststart",
"dead_space_removal": "only verified long silence plus transcript/chat break evidence",
"default_video_overlap_seconds": 0,
"semantic_boundary_overlap_seconds": 5,
"qa_boundary_overlap_seconds": 15,
"chat_carry_forward": true,
"dedupe_final_materials": true,
"large_binary_git_policy": "local-only-unless-explicitly-approved"
},
"outputs": {
"student_packet": true,
"keyframes": true,
"contact_sheets": true,
"verification_reports": true
}
}
| Mode | Typical input | Segmentation | Transcript/chat | Keyframes | Gemini video pass | Output |
|---|---|---|---|---|---|---|
| Long workshop | 2-8 hour Zoom/workshop recording | Required | Required if available | Required | Required | Full student packet |
| Medium tutorial | 20-60 minute lesson | Optional semantic chunks | Transcript required, chat optional | Recommended | Required when deriving facts from video | Guide, chapter index, resources |
| Short bug reproduction | 30 sec-8 min screen recording | Skip | Optional | Required | Required for observed behavior | Bug report, repro steps, implementation brief |
| Quick design feedback | 1-12 min screen and voice | Skip | Recommended if voiceover matters | Required | Required for visual/audio claims | Design-change brief, QA checklist |
| Raw app walkthrough | 5-30 min app demo | Skip under 12 min; segment over 20 min | Recommended | Required | Required for requirements/QA extraction | Requirements doc, acceptance criteria |
"Optional" applies only to extra Oracle planning/review over text artifacts. It does not mean the video-watching step can be skipped when the deliverable depends on what is visible or audible in the recording.
Segment when any of these are true:
Skip segmentation when all of these are true:
Use a medium path when the video is 12-45 minutes, has no real breaks, and has 2-4 obvious sections. In that case, keep one physical video and create timestamped sections unless model upload/runtime gets awkward.
Remove a video span only when long silence or inactive-screen evidence, transcript evidence, and chat evidence agree that the span is a break or dead space. Never discard chat from a removed span. Save it in break-chat/ and carry it forward to the next spoken chunk when it contains a question, answer, link, or context used later.
Extract keyframes when screen share, terminal commands, browser pages, demos, design feedback, or visual claims appear. Skip keyframes only for pure audio/talking-head content where visual evidence cannot change the output.
Run an omissions pass when coverage is incomplete, missing URLs exist, missing chat questions exist, keyframe candidates are thin, machine JSON is missing/unparseable, timestamps are out of range, or hallucination checks find invented tools/links/commands.
cd ~/dev/workshop-video-followup-pipeline
RUN_ID="$(date +%Y-%m-%d)-example-workshop"
RUN_DIR="runs/$RUN_ID"
mkdir -p "$RUN_DIR"/{inputs,receipts,analysis,segments,break-chat,chunks,oracle/sessions,final,publication}
git status --short > "$RUN_DIR/receipts/git-status.before.txt"
Record tool versions:
{
echo "# Tool Versions"
echo
echo "date: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo
echo "## ffmpeg"
ffmpeg -version | head -n 3 || true
echo
echo "## ffprobe"
ffprobe -version | head -n 3 || true
echo
echo "## git"
git --version || true
echo
echo "## jq"
jq --version || true
echo
echo "## python"
python3 --version || true
} > "$RUN_DIR/receipts/tool-versions.txt"
Copy inputs. Do not mutate originals:
cp "/absolute/path/to/source-video.mp4" "$RUN_DIR/inputs/source-video.mp4"
# Optional
cp "/absolute/path/to/transcript.vtt" "$RUN_DIR/inputs/transcript.vtt"
cp "/absolute/path/to/chat.txt" "$RUN_DIR/inputs/chat.txt"
Hash source assets and save file list:
shasum -a 256 "$RUN_DIR/inputs/source-video.mp4" > "$RUN_DIR/inputs/source-video.sha256"
find "$RUN_DIR/inputs" -maxdepth 2 -type f -print0 \
| xargs -0 shasum -a 256 \
> "$RUN_DIR/receipts/source-file-list.txt"
Probe the source video:
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/inputs/source-video.mp4" \
> "$RUN_DIR/receipts/ffprobe.source.json"
Create a human-readable media receipt:
{
echo "# Media Info"
echo
echo "source: $RUN_DIR/inputs/source-video.mp4"
echo "sha256: $(cut -d ' ' -f1 "$RUN_DIR/inputs/source-video.sha256")"
echo
jq -r '.format | "- duration: \(.duration)\n- size: \(.size)\n- bit_rate: \(.bit_rate)"' \
"$RUN_DIR/receipts/ffprobe.source.json"
echo
echo "## Streams"
jq -r '.streams[] | "- index=\(.index) type=\(.codec_type) codec=\(.codec_name) duration=\(.duration // "unknown") width=\(.width // "-") height=\(.height // "-")"' \
"$RUN_DIR/receipts/ffprobe.source.json"
} > "$RUN_DIR/receipts/media-info.md"
Commit the initial evidence:
git add "$RUN_DIR/run-config.json" "$RUN_DIR/inputs" "$RUN_DIR/receipts"
git commit -m "Capture source evidence for $RUN_ID"
ffmpeg -hide_banner -i "$RUN_DIR/inputs/source-video.mp4" \
-af silencedetect=noise=-35dB:d=5 \
-f null - 2> "$RUN_DIR/analysis/silencedetect-35db-5s.log"
Draft analysis/boundary-candidates.md from silence, transcript, and chat evidence.
Create segments/segment-manifest.v1.json and .md. Preserve original timestamps.
Define semantic chapters under 30 minutes for the canonical teaching structure, but
define separate technical analysis shards no longer than four minutes until a
source-specific production-shard probe proves a longer duration safe. Map each shard
back to its canonical chapter. Remove break spans only when evidence supports removal.
Run $oracle before finalizing segmentation when the run is long, public, or ambiguous. Bundle only text artifacts:
mkdir -p ~/.oracle/bundles
find "$RUN_DIR" -type f \( -name '*.md' -o -name '*.json' -o -name '*.log' -o -name '*.sh' \) \
| xargs packx --preview --no-interactive
find "$RUN_DIR" -type f \( -name '*.md' -o -name '*.json' -o -name '*.log' -o -name '*.sh' \) \
| xargs packx --limit 900k -f markdown --no-interactive --stdout --strip-comments --minify \
> ~/.oracle/bundles/video-followup-segmentation-review.txt
test -s ~/.oracle/bundles/video-followup-segmentation-review.txt
Ask Oracle in browser mode only. If using the skill, invoke $oracle; if using the CLI fallback, keep browser mode explicit:
oracle --engine browser \
--model gpt-5.6-pro \
--browser-thinking-time extended \
--slug video-followup-segmentation-review \
--file ~/.oracle/bundles/video-followup-segmentation-review.txt \
-p "$(cat "$RUN_DIR/oracle/planning.md")"
Save the Oracle conversation URL, local bundle path, prompt, and ~/.oracle/sessions/<slug>/output.log reference in oracle/sessions/<slug>.md.
Revise into segments/segment-manifest.v2.json and .md.
Cut archival chapters and Gemini analysis shards from the original source. A
stream-copy cut is acceptable only as an archival artifact; do not send stream-copy
chunks to Gemini by default. A file can pass ffprobe and still stall indefinitely
in Gemini.
mkdir -p "$RUN_DIR/chunks/01"
START_SECONDS=344
DURATION_SECONDS=240
SOURCE="$RUN_DIR/inputs/source-video.mp4"
ANALYSIS_SHARD="$RUN_DIR/chunks/01/chunk.mp4"
ffmpeg -hide_banner -loglevel error -y \
-ss "$START_SECONDS" -i "$SOURCE" -t "$DURATION_SECONDS" \
-map 0:v:0 -map 0:a:0 \
-vf 'fps=30000/1001,format=yuv420p' \
-c:v libx264 -preset veryfast -crf 30 -profile:v main \
-c:a aac -b:a 80k -ar 44100 -ac 2 \
-movflags +faststart "$ANALYSIS_SHARD"
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/chunks/01/chunk.mp4" \
> "$RUN_DIR/chunks/01/chunk.ffprobe.json"
Keep every shard below the configured Oracle attachment ceiling. When the ceiling is
10 MiB, target <=9 MiB. Avoid low-fidelity derivatives such as 4 fps video or 24 kHz
mono audio: they may upload, then become This file has been removed or unreadable.
After any encoding change, rerun the source-specific audio+visual probe before batching.
Slice transcript and chat into chunks/<nn>/transcript.md, chat.md, and carry-forward-chat.md.
Create chunks/video-manifest.json with chunk id, original range, file path, transcript path, chat path, carry-forward path, ffprobe path, and verification status.
Use this path for quick screen captures, short bug reproductions, design notes, and small app walkthroughs.
RUN_ID="$(date +%Y-%m-%d)-quick-clip"
RUN_DIR="runs/$RUN_ID"
mkdir -p "$RUN_DIR"/{inputs,receipts,chunks/01,final}
cp "/absolute/path/to/clip.mov" "$RUN_DIR/inputs/source-video.mov"
shasum -a 256 "$RUN_DIR/inputs/source-video.mov" > "$RUN_DIR/inputs/source-video.sha256"
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/inputs/source-video.mov" \
> "$RUN_DIR/receipts/ffprobe.source.json"
For short clips, use the source directly only after ffprobe plus the source-specific
audio+visual probe confirms Gemini can retain and read it. Otherwise create the same
H.264 Main/yuv420p/AAC-LC/faststart analysis encode used by the long workflow. Do not
downsample to 4 fps or 24 kHz mono merely to reduce size.
ln -s ../../inputs/source-video.mov "$RUN_DIR/chunks/01/chunk.mov"
Create chunks/01/gemini-prompt.md using the short clip prompt in this doc, attach the clip directly to Gemini/video model, and save:
chunks/01/expected-sentinel.txt
chunks/01/gemini-run.log
chunks/01/gemini-result.json
chunks/01/gemini-analysis.md
chunks/01/final-reconciled.md
final/README.md
Final output by short mode:
final/bug-report.md, final/repro-steps.md, final/acceptance-checks.mdfinal/design-brief.md, final/visual-acceptance-criteria.mdfinal/requirements.md, final/qa-checklist.mdfinal/summary.md, final/resources-and-links.mdRun the same malformed URL and hallucination checks even for short clips.
For YouTube research batches, add these source-selection rules before downloading:
upload_date), not title text.upload_date >= 20260501.queue.json with id, title, uploader,
upload_date, duration, and url.yt-dlp when YouTube challenges require it.For each batch item, use a topic-specific prompt. Do not copy old prompt text from a different topic without replacing the product/domain language.
Before a large batch, run two source-specific attached-video probes: a 30-60 second source-native sample, then one production-size <=4 minute analysis shard using the real prompt shape. Each prompt must request one spoken phrase and one visual observation at a specified timestamp, plus an alphabetic-ending sentinel. Independently verify the phrase against audio/captions and the visual observation against an extracted frame. Do not rely on a sentinel-only or text-only probe.
Required preflight receipts:
user-query retaining its attachment chip/duration, with the prompt
absent from the composer and a model-response begun;On any send not accepted, prompt-empty, consent modal, sign-in, quota, rate-limit,
or upload warning, stop new batch starts. Inspect the active Gemini tab, classify the
state, and retry only the affected chunk if no valid sentinel response exists.
Write the exact expected sentinel into chunks/<nn>/expected-sentinel.txt before each
Gemini run. Verification and recovery should compare against that file, not infer the
sentinel shape from memory. End the sentinel with letters, for example
END_OF_VIDEO_ANALYSIS workshop SHARD_004_COMPLETE, not a bare number that may be
visually joined to a following timestamp.
Oracle log lines such as Attachment upload complete, Sending prompt, and still generating are diagnostics, not proof. The live Gemini DOM is authoritative. A title,
sidebar/body text match, empty composer, or broad Stop button match does not prove a
submitted turn. Re-check state before long waits because an apparent turn can regress
and return to the composer.
For liveness, snapshot the real model-response message-content text/HTML and
aria-busy state 2-3 times over about three minutes. If content grows, keep waiting. If
an Analyzing/Gemini said placeholder does not grow, record a confirmed stall, close
only that exact target, frame-exact re-encode the shard, and retry once. Do not loop the
same media bytes.
On a fresh /app conversation, the upload menu can initially be empty. Initialize the
conversation with one harmless text turn, wait for completion, reopen the upload menu,
then run the actual source-specific video probe. The initializer is not video proof.
When using Oracle CLI fallback with --browser-keep-browser, cleanup is mandatory:
gemini-result.json, gemini-analysis.md, gemini-analysis.extracted.json when
expected, machine JSON, and any recovery note if the response was recovered by tab
inspection.END_OF_VIDEO_ANALYSIS ... sentinel.For scripted runs, implement cleanup as a post-extraction step, not as a generic
finally block. A safe script order is: save wrapper result, inspect/recover tab output
if needed, write extracted JSON, verify the exact expected sentinel, then close the
specific Gemini tab target. If any extraction or sentinel check fails, leave the tab
open and record the Chrome port/target for recovery.
If exact target close tooling is available, use Chrome DevTools Protocol to close the
specific gemini.google.com/app target created for the run. If not, list tabs first,
identify completed or empty Gemini tabs, and close only those. Avoid broad "close all
Gemini" actions while a batch is running.
The Chrome target id returned by /json/list is different from the Gemini conversation
id in /app/<conversation-id>. Close /json/close/<target-id>, then list targets again
to prove removal. Do not pass the conversation-id suffix to the close endpoint.
When classifying stale tabs:
You are analyzing one source-preserving video chunk for a follow-up packet.
Run metadata:
- run_key: {{run_key}}
- chunk_id: {{chunk_id}}
- original_recording_range: {{start_time}} -> {{end_time}}
- chunk_video: {{chunk_video_path}}
- transcript: included below
- chat: included below
- carry_forward_chat: included below
- next_chunk_preview: included below
Hard rules:
1. Preserve original recording timestamps.
2. Separate transcript evidence, chat evidence, visual evidence, and inference.
3. Do not invent tools, commands, URLs, files, speakers, or UI labels.
4. Include unanswered and deferred questions.
5. Flag every policy-sensitive area, especially browser automation, cookies, quota/rate limits, private accounts, and publishing.
6. Return both Markdown and fenced JSON.
7. End with: END_OF_VIDEO_ANALYSIS {{run_key}} {{chunk_id}}_COMPLETE
Return:
# Chunk Analysis: {{chunk_id}}
## 1. Overview
## 2. Dense Timeline
## 3. Concept Inventory
## 4. Demo / Screen Activity
## 5. Questions And Answers
## 6. Links / Tools / Commands / References
## 7. Keyframes To Capture
## 8. Student Exercises
## 9. Uncertainties / Needs Review
## 10. Machine JSON
You are auditing a prior video analysis for omissions.
Inputs:
- chunk_id: {{chunk_id}}
- original_recording_range: {{start_time}} -> {{end_time}}
- coverage_report: {{coverage_report}}
- expected_items: {{expected_items_json}}
- prior_analysis: {{prior_analysis}}
Find only omissions and corrections. Do not repeat complete items.
Required focus:
1. Missing URLs exactly as they appeared.
2. Missing chat questions, including unanswered/deferred questions.
3. Missing demo steps.
4. Missing keyframe candidates.
5. Unsupported visual claims.
6. Hallucinated tools, commands, links, or files.
7. Parseable fenced JSON.
Return:
# Omissions And Corrections: {{chunk_id}}
## Additions
## Corrections
## Remove Or Downgrade
## Remaining Human Review
## Machine JSON
Reconcile one analyzed chunk into an evidence-grounded final chunk record.
Inputs:
- segment manifest entry
- transcript slice
- chat slice
- carry-forward chat
- main analysis
- omissions analysis
- verification reports
- keyframe review
Rules:
1. Preserve original timestamps.
2. Remove malformed URLs.
3. Do not include any URL/tool/command unless supported by transcript, chat, carry-forward chat, local path, or keyframe review.
4. Do not claim a visual is visible unless keyframe review confirms it.
5. Preserve unanswered/deferred questions.
6. Preserve editorial warnings.
7. Deduplicate overlap by normalized timestamp + item type + text hash.
Return Markdown and JSON with:
- status: PASS | PASS_WITH_WARNINGS | FAIL
- counts
- timeline summary
- concepts
- Q&A
- links/tools/commands
- demos
- keyframes
- warnings
- student-material candidates
You are analyzing a short screen recording or app walkthrough.
Source:
- run_key: {{run_key}}
- recording_type: short_clip | bug_reproduction | design_feedback | app_walkthrough
- original_recording_range: 00:00:00 -> {{duration}}
- source_video_sha256: {{source_video_sha256}}
Hard rules:
1. Use timestamps relative to the clip unless an original recording timeline is provided.
2. Do not invent commands, files, UI labels, URLs, or implementation details.
3. Separate observed behavior from inferred intent.
4. Include timestamped screenshot/keyframe candidates for visual claims.
5. For a bug reproduction, produce repro steps and expected vs actual behavior.
6. For design feedback, produce requested changes and visual acceptance criteria.
7. For an app walkthrough, produce requirements and QA checks.
8. Include uncertainties and manual review points.
Return:
# Short Video Analysis: {{run_key}}
## 1. Summary
## 2. Observed Timeline
## 3. User Intent / Requirements
## 4. Reproduction Steps Or Walkthrough Steps
## 5. Expected Vs Actual Behavior
## 6. Implementation Notes
## 7. QA / Acceptance Criteria
## 8. Keyframes To Extract
## 9. Risks / Unknowns
## 10. Machine JSON
Assemble a student follow-up packet from verified chunk records.
Inputs:
- segments/segment-manifest.v2.md
- chunks/*/final-reconciled.md
- chunks/*/final-reconciled.json
- chunks/*/verification/*.md
- final/keyframes-manifest.json if present
- resources extracted from chat/transcript
Rules:
1. Use original recording timestamps.
2. Do not include unverified visual claims.
3. Do not include malformed URLs.
4. Do not present experimental tools as official recommendations unless approved.
5. Keep policy-sensitive automation/cookie/quota/rate-limit sections as editorial notes.
6. Distinguish publication-ready files from reference drafts.
7. Deduplicate overlap and carry-forward repeated material.
Create:
- final/README.md
- final/student-followup-guide.md
- final/chapter-index.md
- final/resources-and-links.md
- final/qa-index.md
- final/exercises.md
- final/keyframes-manifest.md
- final/keyframes-manifest.json
- final/instructor-review-checklist.md
[{{slug}}]
Project briefing:
{{project_briefing}}
Goal:
{{goal}}
Prior-State Ledger:
- prior_oracle_session: {{prior_oracle_session_or_none}}
- last_oracle_recommendation: {{summary_or_none}}
- implemented_since_prior_oracle: {{files_or_none}}
- verified_since_prior_oracle: {{commands_or_none}}
- failed_proof_or_blocker: {{receipt_or_none}}
- narrowing_decision: {{one_slice}}
- files_functions_now_in_scope: {{paths}}
- files_functions_out_of_scope: {{paths}}
- next_falsifiable_verification: {{command_or_receipt}}
- forward_progress_index: {{score_and_arithmetic}}
Deliverable:
Evaluate segmentation, chat carry-forward, analysis prompts, verification gates, final packet readiness, privacy/security risks, and the one highest-leverage next local action.
Return your answer as text in this response only. Do not create, attach, export, or offer any downloadable file. Do not create local project artifacts yourself. The local agent will write any needed files, plans, notes, goals, commits, or verification logs using local tools.
Treat ~/.oracle/sessions/<slug>/output.log, browser logs, and raw Gemini responses as
private evidence. The MCP return can be truncated, so recover the full response from
the persisted session log before reconciliation. Before copying any material into
publication/public-draft/ or another public wiki, scan and redact credentials,
auth/callback URLs, account/model identifiers, local usernames and paths,
signed/private URLs, notifications, and personal data. Gemini can repeat these even
when told not to.
Every pull quote and timestamp must be checked against local audio/captions and the source frame or clip. Gemini-supplied quote timestamps are candidates, not receipts. Record the verified timestamp and evidence path in the reconciliation artifact.
Run malformed URL checks on every final packet:
rg -n 'https://www\.\.\.|https://www\s|/"|\| .*: \||Context:' "$RUN_DIR/final" || true
Extract exact URLs from chat/transcript and compare against final resources:
rg -o 'https?://[^[:space:])>"'"'"']+' "$RUN_DIR/chunks" "$RUN_DIR/break-chat" \
| sed 's/[,:;.!"]$//' \
| sort -u \
> "$RUN_DIR/verification.expected-urls.txt"
rg -o 'https?://[^[:space:])>"'"'"']+' "$RUN_DIR/final/resources-and-links.md" \
| sed 's/[,:;.!"]$//' \
| sort -u \
> "$RUN_DIR/verification.final-urls.txt"
comm -23 "$RUN_DIR/verification.expected-urls.txt" "$RUN_DIR/verification.final-urls.txt" \
> "$RUN_DIR/verification.missing-urls.txt"
Validate timestamps stay within range, except carry-forward chat:
python3 scripts/validate-timestamps.py "$RUN_DIR" \
> "$RUN_DIR/receipts/timestamp-validation.log"
If the script does not exist yet, perform a manual timestamp pass and save it as chunks/<nn>/verification/timestamp-report.md.
Check machine JSON:
find "$RUN_DIR/chunks" -name '*.json' -print0 | xargs -0 -n1 jq empty
Required chunk gate files:
chunks/<nn>/verification/coverage-report.md
chunks/<nn>/verification/hallucination-report.md
chunks/<nn>/verification/timestamp-report.md
chunks/<nn>/verification/qa-report.md
chunks/<nn>/final-reconciled.md
chunks/<nn>/final-reconciled.json
Chunk status meanings:
PASS: complete and publication-safe after ordinary instructor review.PASS_WITH_WARNINGS: usable evidence, but warnings must be reviewed before publication.FAIL: rerun analysis, omissions, or reconciliation before using the chunk.Candidate extraction prompt output should include timestamp, reason, expected visual, priority, and confidence.
Extract deterministic frames:
mkdir -p "$RUN_DIR/chunks/01/keyframes/extracted"
ffmpeg -hide_banner -ss 00:23:48 \
-i "$RUN_DIR/inputs/source-video.mp4" \
-frames:v 1 \
"$RUN_DIR/chunks/01/keyframes/extracted/00-23-48-automation-verification.jpg"
Create contact sheet:
find "$RUN_DIR/chunks/01/keyframes/extracted" -name '*.jpg' | sort \
> "$RUN_DIR/chunks/01/keyframes/contact-sheet-inputs.txt"
ffmpeg -hide_banner -f concat -safe 0 \
-i <(sed "s#^#file '#; s#$#'#" "$RUN_DIR/chunks/01/keyframes/contact-sheet-inputs.txt") \
-vf "scale=320:-1,tile=3x4" \
"$RUN_DIR/chunks/01/keyframes/contact-sheet.jpg"
If process substitution is unavailable, generate the concat list as a real file and pass it to ffmpeg.
Initialize a new artifact project:
mkdir -p ~/dev/new-video-followup-project
cd ~/dev/new-video-followup-project
git init
cat > README.md <<'EOF'
# Video Follow-Up Pipeline
Local artifact project for repeatable video follow-up runs.
EOF
git add README.md
git commit -m "Initialize video follow-up project"
Use proof-oriented commit boundaries:
git commit -m "Capture source evidence for <run_id>"
git commit -m "Draft initial segmentation for <run_id>"
git commit -m "Record Oracle segmentation review for <run_id>"
git commit -m "Revise semantic chunk manifest"
git commit -m "Cut video chunks and ffprobe receipts"
git commit -m "Slice transcript and chat by chunk"
git commit -m "Run Gemini analysis for chunks 01-03"
git commit -m "Add verification reports for chunks 01-03"
git commit -m "Run omissions pass for chunk 01"
git commit -m "Reconcile verified chunk analyses"
git commit -m "Extract and review keyframe candidates"
git commit -m "Assemble student follow-up packet"
git commit -m "Document repeatable video follow-up process"
Default binary policy for future runs:
Suggested .gitignore for future automation projects:
runs/**/inputs/*.mp4
runs/**/inputs/*.mov
runs/**/inputs/*.mkv
runs/**/chunks/**/*.mp4
runs/**/chunks/**/*.mov
!runs/**/inputs/*.sha256
!runs/**/receipts/**
!runs/**/segments/**
!runs/**/chunks/**/*.json
!runs/**/chunks/**/*.md
!runs/**/chunks/**/*.log
!runs/**/final/**
!runs/**/oracle/**
runs/**/chunks/**/keyframes/extracted/*.jpg
runs/**/chunks/**/keyframes/extracted/*.png
runs/**/chunks/**/keyframes/strips/*.jpg
runs/**/chunks/**/keyframes/contact-sheet*.jpg
If committing binaries intentionally:
git add -f "$RUN_DIR/chunks/01/chunk.mp4"
git add -f "$RUN_DIR/chunks/01/keyframes/contact-sheet.jpg"
For every $oracle call, save:
- slug
- date
- run id
- browser mode: true
- ChatGPT conversation URL
- bundle path
- bundle sha256
- prompt path
- output log path
- local decisions accepted/rejected
- commit before
- commit after
Template:
# Oracle Session: {{session_slug}}
- date:
- local_run:
- browser_mode: true
- session_url:
- bundle_path:
- bundle_sha256:
- prompt_file:
- output_file:
- commit_before:
- commit_after:
## Question
## Oracle Response Summary
## Local Decisions
Accepted:
Rejected:
Follow-up:
Treat these as first publication candidates:
final/README.mdfinal/student-followup-guide.mdfinal/chapter-index.mdfinal/resources-and-links.mdfinal/instructor-review-checklist.mdTreat these as reference drafts until edited:
final/qa-index.mdfinal/exercises.mdfinal/keyframes-manifest.mdManual approval is mandatory for:
Project setup:
inputs/.Long recording preprocessing:
silencedetect.segment-manifest.v1.json and .md.$oracle.segment-manifest.v2.json and .md.Chunk creation:
ffprobe every chunk.chunks/video-manifest.json.Chunk analysis:
user-query, retained attachment, prompt absent from composer, begun/completed model-response) and an alphabetic-ending END_OF_VIDEO_ANALYSIS sentinel; wrapper logs alone are not proof.model-response message-content growth; classify a no-growth placeholder over 2-3 snapshots/~3 minutes as stalled, re-encode, and retry once.expected-items.json.Visual evidence:
Reconciliation:
final-reconciled.md/json per chunk.Final packet:
publication/private-redactions.md.Process documentation:
docs/repeatable-video-followup-process.md.Do not build these until the process doc is reviewed:
scripts/video-followup/init-run.ts
scripts/video-followup/probe-source.ts
scripts/video-followup/detect-silence.ts
scripts/video-followup/build-segment-manifest.ts
scripts/video-followup/cut-chunks.ts
scripts/video-followup/slice-transcript.ts
scripts/video-followup/slice-chat.ts
scripts/video-followup/build-gemini-prompts.ts
scripts/video-followup/extract-machine-json.ts
scripts/video-followup/verify-coverage.ts
scripts/video-followup/scan-malformed-urls.ts
scripts/video-followup/extract-keyframes.ts
scripts/video-followup/build-contact-sheet.ts
scripts/video-followup/reconcile-chunk.ts
scripts/video-followup/build-final-packet.ts
templates/gemini/chunk-analysis.md
templates/gemini/omissions.md
templates/oracle/planning-review.md
templates/oracle/final-review.md
templates/final/README.md
templates/final/student-followup-guide.md
templates/final/chapter-index.md
templates/final/resources-and-links.md
templates/final/instructor-review-checklist.md
docs/automation-roadmap.md
Automation priority:
# Repeatable Video Follow-Up Process
Status: implementation-ready process spec
Owner: local coding agent
Reference run: `workshop-followup/`
Primary output: verified follow-up materials from one or more source videos
This process turns a raw workshop recording, tutorial, app walkthrough, bug reproduction, or short screen recording into a reviewable artifact project. It is built from the completed `workshop-followup/` run and the Oracle review session `video-pipeline-repeatable-process`.
The pipeline has two main paths:
1. Long workshop path: segment long recordings, remove verified breaks, carry forward break chat, analyze chunks, verify omissions, extract keyframes, reconcile, and assemble a student packet.
2. Short recording path: skip segmentation unless needed, analyze the clip directly, optionally extract keyframes, verify outputs, and produce a compact follow-up note, bug packet, design brief, or requirements document.
The core rule is evidence preservation. Keep the original media immutable, preserve original timestamps when a source timeline exists, save model prompts/results/logs, and keep verification receipts next to every generated claim.
Operational rule from the 2026-06 YouTube batch runs: Gemini video analysis must use
Oracle browser mode and must clean up after itself. After a chunk's analysis is saved
and verified, close the Gemini tab opened for that chunk. Leave the browser/profile
alive for reuse, but do not leave accumulated `gemini.google.com/app` tabs on the
remote Mac mini.
## Project Structure
Use one run folder per video or video set:
```text
runs/
YYYY-MM-DD-slug/
README.md
run-config.json
inputs/
source-video.mp4
source-video.sha256
transcript.vtt
transcript.md
chat.txt
chat.md
screenshots/
clips/
receipts/
ffprobe.source.json
media-info.md
source-file-list.txt
tool-versions.txt
git-status.before.txt
git-status.after.txt
analysis/
timeline-initial.json
silencedetect.log
boundary-candidates.md
segmentation-notes.md
source-timeline.md
segments/
segment-manifest.v1.json
segment-manifest.v1.md
segment-manifest.v2.json
segment-manifest.v2.md
break-chat/
morning-break.chat.md
lunch-break.chat.md
other-break.chat.md
chunks/
video-manifest.json
01/
chunk.mp4
chunk.ffprobe.json
transcript.md
chat.md
carry-forward-chat.md
next-chunk-preview.md
gemini-prompt.md
expected-sentinel.txt
gemini-run.log
gemini-result.json
gemini-analysis.md
gemini-analysis.extracted.json
gemini-omissions-prompt.md
gemini-omissions-run.log
gemini-omissions-result.json
gemini-omissions.md
final-reconciled.md
final-reconciled.json
keyframes/
candidates.json
extracted/
strips/
contact-sheet.jpg
review.md
verification/
expected-items.json
extracted-items.json
coverage-report.md
hallucination-report.md
timestamp-report.md
qa-report.md
visual-claim-report.md
oracle/
planning.md
review.md
sessions/
final/
README.md
student-followup-guide.md
chapter-index.md
resources-and-links.md
qa-index.md
exercises.md
keyframes-manifest.md
keyframes-manifest.json
instructor-review-checklist.md
publication/
public-draft/
private-redactions.md
publish-checklist.md
```
The current reference artifacts live in `workshop-followup/`. New runs should use `runs/<date>-<slug>/` so the reference run remains untouched.
## Run Config
Create `run-config.json` before processing:
```json
{
"run_id": "2026-05-30-example-workshop",
"title": "Example Workshop",
"mode": "long_workshop",
"source": {
"video": "inputs/source-video.mp4",
"transcript": "inputs/transcript.vtt",
"chat": "inputs/chat.txt",
"screenshots": "inputs/screenshots",
"clips": "inputs/clips"
},
"policy": {
"preserve_original_timestamps": true,
"canonical_chapters": "semantic",
"canonical_chapter_max_minutes": 30,
"analysis_shard_target_minutes": 4,
"analysis_shard_max_minutes": 4,
"analysis_attachment_target_mib": 9,
"analysis_media": "h264-main-yuv420p-29.97fps-aac-lc-44.1khz-faststart",
"dead_space_removal": "only verified long silence plus transcript/chat break evidence",
"default_video_overlap_seconds": 0,
"semantic_boundary_overlap_seconds": 5,
"qa_boundary_overlap_seconds": 15,
"chat_carry_forward": true,
"dedupe_final_materials": true,
"large_binary_git_policy": "local-only-unless-explicitly-approved"
},
"outputs": {
"student_packet": true,
"keyframes": true,
"contact_sheets": true,
"verification_reports": true
}
}
```
## Mode Matrix
| Mode | Typical input | Segmentation | Transcript/chat | Keyframes | Gemini video pass | Output |
|---|---|---:|---:|---:|---:|---|
| Long workshop | 2-8 hour Zoom/workshop recording | Required | Required if available | Required | Required | Full student packet |
| Medium tutorial | 20-60 minute lesson | Optional semantic chunks | Transcript required, chat optional | Recommended | Required when deriving facts from video | Guide, chapter index, resources |
| Short bug reproduction | 30 sec-8 min screen recording | Skip | Optional | Required | Required for observed behavior | Bug report, repro steps, implementation brief |
| Quick design feedback | 1-12 min screen and voice | Skip | Recommended if voiceover matters | Required | Required for visual/audio claims | Design-change brief, QA checklist |
| Raw app walkthrough | 5-30 min app demo | Skip under 12 min; segment over 20 min | Recommended | Required | Required for requirements/QA extraction | Requirements doc, acceptance criteria |
"Optional" applies only to extra Oracle planning/review over text artifacts. It does
not mean the video-watching step can be skipped when the deliverable depends on what
is visible or audible in the recording.
## Decision Tree
Segment when any of these are true:
- Video duration is over 30 minutes.
- There are real breaks, lunch, pre-roll, or dead air.
- Q&A spans topic transitions.
- Chat must be aligned to the video.
- The target output is a course packet, student packet, chapter index, or publication bundle.
Skip segmentation when all of these are true:
- Video duration is 12 minutes or less.
- There is no long dead air or break.
- Chat is absent, irrelevant, or small enough to include directly.
- One model pass can analyze the whole video reliably.
- The target output is a short note, bug report, design feedback memo, or compact implementation plan.
Use a medium path when the video is 12-45 minutes, has no real breaks, and has 2-4 obvious sections. In that case, keep one physical video and create timestamped sections unless model upload/runtime gets awkward.
Remove a video span only when long silence or inactive-screen evidence, transcript evidence, and chat evidence agree that the span is a break or dead space. Never discard chat from a removed span. Save it in `break-chat/` and carry it forward to the next spoken chunk when it contains a question, answer, link, or context used later.
Extract keyframes when screen share, terminal commands, browser pages, demos, design feedback, or visual claims appear. Skip keyframes only for pure audio/talking-head content where visual evidence cannot change the output.
Run an omissions pass when coverage is incomplete, missing URLs exist, missing chat questions exist, keyframe candidates are thin, machine JSON is missing/unparseable, timestamps are out of range, or hallucination checks find invented tools/links/commands.
## Project Setup
```bash
cd ~/dev/workshop-video-followup-pipeline
RUN_ID="$(date +%Y-%m-%d)-example-workshop"
RUN_DIR="runs/$RUN_ID"
mkdir -p "$RUN_DIR"/{inputs,receipts,analysis,segments,break-chat,chunks,oracle/sessions,final,publication}
git status --short > "$RUN_DIR/receipts/git-status.before.txt"
```
Record tool versions:
```bash
{
echo "# Tool Versions"
echo
echo "date: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo
echo "## ffmpeg"
ffmpeg -version | head -n 3 || true
echo
echo "## ffprobe"
ffprobe -version | head -n 3 || true
echo
echo "## git"
git --version || true
echo
echo "## jq"
jq --version || true
echo
echo "## python"
python3 --version || true
} > "$RUN_DIR/receipts/tool-versions.txt"
```
Copy inputs. Do not mutate originals:
```bash
cp "/absolute/path/to/source-video.mp4" "$RUN_DIR/inputs/source-video.mp4"
# Optional
cp "/absolute/path/to/transcript.vtt" "$RUN_DIR/inputs/transcript.vtt"
cp "/absolute/path/to/chat.txt" "$RUN_DIR/inputs/chat.txt"
```
Hash source assets and save file list:
```bash
shasum -a 256 "$RUN_DIR/inputs/source-video.mp4" > "$RUN_DIR/inputs/source-video.sha256"
find "$RUN_DIR/inputs" -maxdepth 2 -type f -print0 \
| xargs -0 shasum -a 256 \
> "$RUN_DIR/receipts/source-file-list.txt"
```
Probe the source video:
```bash
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/inputs/source-video.mp4" \
> "$RUN_DIR/receipts/ffprobe.source.json"
```
Create a human-readable media receipt:
```bash
{
echo "# Media Info"
echo
echo "source: $RUN_DIR/inputs/source-video.mp4"
echo "sha256: $(cut -d ' ' -f1 "$RUN_DIR/inputs/source-video.sha256")"
echo
jq -r '.format | "- duration: \(.duration)\n- size: \(.size)\n- bit_rate: \(.bit_rate)"' \
"$RUN_DIR/receipts/ffprobe.source.json"
echo
echo "## Streams"
jq -r '.streams[] | "- index=\(.index) type=\(.codec_type) codec=\(.codec_name) duration=\(.duration // "unknown") width=\(.width // "-") height=\(.height // "-")"' \
"$RUN_DIR/receipts/ffprobe.source.json"
} > "$RUN_DIR/receipts/media-info.md"
```
Commit the initial evidence:
```bash
git add "$RUN_DIR/run-config.json" "$RUN_DIR/inputs" "$RUN_DIR/receipts"
git commit -m "Capture source evidence for $RUN_ID"
```
## Long Recording Workflow
1. Detect silence:
```bash
ffmpeg -hide_banner -i "$RUN_DIR/inputs/source-video.mp4" \
-af silencedetect=noise=-35dB:d=5 \
-f null - 2> "$RUN_DIR/analysis/silencedetect-35db-5s.log"
```
2. Draft `analysis/boundary-candidates.md` from silence, transcript, and chat evidence.
3. Create `segments/segment-manifest.v1.json` and `.md`. Preserve original timestamps.
Define semantic chapters under 30 minutes for the canonical teaching structure, but
define separate technical analysis shards no longer than four minutes until a
source-specific production-shard probe proves a longer duration safe. Map each shard
back to its canonical chapter. Remove break spans only when evidence supports removal.
4. Run `$oracle` before finalizing segmentation when the run is long, public, or ambiguous. Bundle only text artifacts:
```bash
mkdir -p ~/.oracle/bundles
find "$RUN_DIR" -type f \( -name '*.md' -o -name '*.json' -o -name '*.log' -o -name '*.sh' \) \
| xargs packx --preview --no-interactive
find "$RUN_DIR" -type f \( -name '*.md' -o -name '*.json' -o -name '*.log' -o -name '*.sh' \) \
| xargs packx --limit 900k -f markdown --no-interactive --stdout --strip-comments --minify \
> ~/.oracle/bundles/video-followup-segmentation-review.txt
test -s ~/.oracle/bundles/video-followup-segmentation-review.txt
```
Ask Oracle in browser mode only. If using the skill, invoke `$oracle`; if using the CLI fallback, keep browser mode explicit:
```bash
oracle --engine browser \
--model gpt-5.6-pro \
--browser-thinking-time extended \
--slug video-followup-segmentation-review \
--file ~/.oracle/bundles/video-followup-segmentation-review.txt \
-p "$(cat "$RUN_DIR/oracle/planning.md")"
```
Save the Oracle conversation URL, local bundle path, prompt, and `~/.oracle/sessions/<slug>/output.log` reference in `oracle/sessions/<slug>.md`.
5. Revise into `segments/segment-manifest.v2.json` and `.md`.
6. Cut archival chapters and Gemini analysis shards from the original source. A
stream-copy cut is acceptable only as an archival artifact; do not send stream-copy
chunks to Gemini by default. A file can pass `ffprobe` and still stall indefinitely
in Gemini.
```bash
mkdir -p "$RUN_DIR/chunks/01"
START_SECONDS=344
DURATION_SECONDS=240
SOURCE="$RUN_DIR/inputs/source-video.mp4"
ANALYSIS_SHARD="$RUN_DIR/chunks/01/chunk.mp4"
ffmpeg -hide_banner -loglevel error -y \
-ss "$START_SECONDS" -i "$SOURCE" -t "$DURATION_SECONDS" \
-map 0:v:0 -map 0:a:0 \
-vf 'fps=30000/1001,format=yuv420p' \
-c:v libx264 -preset veryfast -crf 30 -profile:v main \
-c:a aac -b:a 80k -ar 44100 -ac 2 \
-movflags +faststart "$ANALYSIS_SHARD"
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/chunks/01/chunk.mp4" \
> "$RUN_DIR/chunks/01/chunk.ffprobe.json"
```
Keep every shard below the configured Oracle attachment ceiling. When the ceiling is
10 MiB, target <=9 MiB. Avoid low-fidelity derivatives such as 4 fps video or 24 kHz
mono audio: they may upload, then become `This file has been removed` or unreadable.
After any encoding change, rerun the source-specific audio+visual probe before batching.
7. Slice transcript and chat into `chunks/<nn>/transcript.md`, `chat.md`, and `carry-forward-chat.md`.
8. Create `chunks/video-manifest.json` with chunk id, original range, file path, transcript path, chat path, carry-forward path, ffprobe path, and verification status.
## Short Recording Workflow
Use this path for quick screen captures, short bug reproductions, design notes, and small app walkthroughs.
```bash
RUN_ID="$(date +%Y-%m-%d)-quick-clip"
RUN_DIR="runs/$RUN_ID"
mkdir -p "$RUN_DIR"/{inputs,receipts,chunks/01,final}
cp "/absolute/path/to/clip.mov" "$RUN_DIR/inputs/source-video.mov"
shasum -a 256 "$RUN_DIR/inputs/source-video.mov" > "$RUN_DIR/inputs/source-video.sha256"
ffprobe -hide_banner -show_format -show_streams -of json \
"$RUN_DIR/inputs/source-video.mov" \
> "$RUN_DIR/receipts/ffprobe.source.json"
```
For short clips, use the source directly only after `ffprobe` plus the source-specific
audio+visual probe confirms Gemini can retain and read it. Otherwise create the same
H.264 Main/yuv420p/AAC-LC/faststart analysis encode used by the long workflow. Do not
downsample to 4 fps or 24 kHz mono merely to reduce size.
```bash
ln -s ../../inputs/source-video.mov "$RUN_DIR/chunks/01/chunk.mov"
```
Create `chunks/01/gemini-prompt.md` using the short clip prompt in this doc, attach the clip directly to Gemini/video model, and save:
```text
chunks/01/expected-sentinel.txt
chunks/01/gemini-run.log
chunks/01/gemini-result.json
chunks/01/gemini-analysis.md
chunks/01/final-reconciled.md
final/README.md
```
Final output by short mode:
- Bug reproduction: `final/bug-report.md`, `final/repro-steps.md`, `final/acceptance-checks.md`
- Design feedback: `final/design-brief.md`, `final/visual-acceptance-criteria.md`
- App walkthrough: `final/requirements.md`, `final/qa-checklist.md`
- Tiny tutorial: `final/summary.md`, `final/resources-and-links.md`
Run the same malformed URL and hallucination checks even for short clips.
For YouTube research batches, add these source-selection rules before downloading:
- Search broadly, but select by actual YouTube metadata (`upload_date`), not title text.
- Enforce user cutoffs exactly. Example: after May 1, 2026 means
`upload_date >= 20260501`.
- Exclude topic collisions, coupon/affiliate uploads, auto-generated music/topic
uploads, and unrelated videos.
- Save selected videos in `queue.json` with `id`, `title`, `uploader`,
`upload_date`, `duration`, and `url`.
- Use Chrome cookies with `yt-dlp` when YouTube challenges require it.
- Space Gemini upload starts by about five minutes for batch runs only after the first
attached-video probe passes and no unresolved run is pending, unless the user asks
for a different cadence.
For each batch item, use a topic-specific prompt. Do not copy old prompt text from a
different topic without replacing the product/domain language.
## Gemini Readiness And Failure Recovery
Before a large batch, run two source-specific attached-video probes: a 30-60 second
source-native sample, then one production-size <=4 minute analysis shard using the real
prompt shape. Each prompt must request one spoken phrase and one visual observation at
a specified timestamp, plus an alphabetic-ending sentinel. Independently verify the
phrase against audio/captions and the visual observation against an extracted frame.
Do not rely on a sentinel-only or text-only probe.
Required preflight receipts:
- configured browser mode and Gemini provider routing;
- visible signed-in Gemini UI and both completed attached-video probes;
- prompt path and exact expected sentinel;
- run/session metadata;
- a real submitted `user-query` retaining its attachment chip/duration, with the prompt
absent from the composer and a `model-response` begun;
- locally verified source-specific audio and visual facts plus the expected sentinel;
- completed tab cleanup.
On any `send not accepted`, `prompt-empty`, consent modal, sign-in, quota, rate-limit,
or upload warning, stop new batch starts. Inspect the active Gemini tab, classify the
state, and retry only the affected chunk if no valid sentinel response exists.
Write the exact expected sentinel into `chunks/<nn>/expected-sentinel.txt` before each
Gemini run. Verification and recovery should compare against that file, not infer the
sentinel shape from memory. End the sentinel with letters, for example
`END_OF_VIDEO_ANALYSIS workshop SHARD_004_COMPLETE`, not a bare number that may be
visually joined to a following timestamp.
Oracle log lines such as `Attachment upload complete`, `Sending prompt`, and `still
generating` are diagnostics, not proof. The live Gemini DOM is authoritative. A title,
sidebar/body text match, empty composer, or broad `Stop` button match does not prove a
submitted turn. Re-check state before long waits because an apparent turn can regress
and return to the composer.
For liveness, snapshot the real `model-response message-content` text/HTML and
`aria-busy` state 2-3 times over about three minutes. If content grows, keep waiting. If
an `Analyzing`/`Gemini said` placeholder does not grow, record a confirmed stall, close
only that exact target, frame-exact re-encode the shard, and retry once. Do not loop the
same media bytes.
On a fresh `/app` conversation, the upload menu can initially be empty. Initialize the
conversation with one harmless text turn, wait for completion, reopen the upload menu,
then run the actual source-specific video probe. The initializer is not video proof.
## Gemini Browser Cleanup
When using Oracle CLI fallback with `--browser-keep-browser`, cleanup is mandatory:
1. Wait for the run to finish.
2. Save every needed extraction artifact from the Gemini response:
`gemini-result.json`, `gemini-analysis.md`, `gemini-analysis.extracted.json` when
expected, machine JSON, and any recovery note if the response was recovered by tab
inspection.
3. Verify the analysis is non-empty and contains the expected
`END_OF_VIDEO_ANALYSIS ...` sentinel.
4. Close only the completed and fully extracted Gemini tab for that run.
5. Do not close tabs for active uploads, still-generating sessions, unresolved wrapper
errors, or completed-looking tabs whose visible response has not been extracted.
6. Do not kill Chrome; Oracle needs the signed-in profile for future runs.
For scripted runs, implement cleanup as a post-extraction step, not as a generic
`finally` block. A safe script order is: save wrapper result, inspect/recover tab output
if needed, write extracted JSON, verify the exact expected sentinel, then close the
specific Gemini tab target. If any extraction or sentinel check fails, leave the tab
open and record the Chrome port/target for recovery.
If exact target close tooling is available, use Chrome DevTools Protocol to close the
specific `gemini.google.com/app` target created for the run. If not, list tabs first,
identify completed or empty Gemini tabs, and close only those. Avoid broad "close all
Gemini" actions while a batch is running.
The Chrome target id returned by `/json/list` is different from the Gemini conversation
id in `/app/<conversation-id>`. Close `/json/close/<target-id>`, then list targets again
to prove removal. Do not pass the conversation-id suffix to the close endpoint.
When classifying stale tabs:
- Empty generic Gemini tabs from failed diagnostics may be closed after inspection.
- Active upload/generation tabs must stay open.
- Tabs with conversation state or auto-generated titles must be checked for saved
output and the expected sentinel before closing.
- Never use broad "close all Gemini tabs" while a batch is active.
## Prompt Templates
### Gemini Chunk Analysis
```markdown
You are analyzing one source-preserving video chunk for a follow-up packet.
Run metadata:
- run_key: {{run_key}}
- chunk_id: {{chunk_id}}
- original_recording_range: {{start_time}} -> {{end_time}}
- chunk_video: {{chunk_video_path}}
- transcript: included below
- chat: included below
- carry_forward_chat: included below
- next_chunk_preview: included below
Hard rules:
1. Preserve original recording timestamps.
2. Separate transcript evidence, chat evidence, visual evidence, and inference.
3. Do not invent tools, commands, URLs, files, speakers, or UI labels.
4. Include unanswered and deferred questions.
5. Flag every policy-sensitive area, especially browser automation, cookies, quota/rate limits, private accounts, and publishing.
6. Return both Markdown and fenced JSON.
7. End with: END_OF_VIDEO_ANALYSIS {{run_key}} {{chunk_id}}_COMPLETE
Return:
# Chunk Analysis: {{chunk_id}}
## 1. Overview
## 2. Dense Timeline
## 3. Concept Inventory
## 4. Demo / Screen Activity
## 5. Questions And Answers
## 6. Links / Tools / Commands / References
## 7. Keyframes To Capture
## 8. Student Exercises
## 9. Uncertainties / Needs Review
## 10. Machine JSON
```
### Omissions Pass
```markdown
You are auditing a prior video analysis for omissions.
Inputs:
- chunk_id: {{chunk_id}}
- original_recording_range: {{start_time}} -> {{end_time}}
- coverage_report: {{coverage_report}}
- expected_items: {{expected_items_json}}
- prior_analysis: {{prior_analysis}}
Find only omissions and corrections. Do not repeat complete items.
Required focus:
1. Missing URLs exactly as they appeared.
2. Missing chat questions, including unanswered/deferred questions.
3. Missing demo steps.
4. Missing keyframe candidates.
5. Unsupported visual claims.
6. Hallucinated tools, commands, links, or files.
7. Parseable fenced JSON.
Return:
# Omissions And Corrections: {{chunk_id}}
## Additions
## Corrections
## Remove Or Downgrade
## Remaining Human Review
## Machine JSON
```
### Local Reconciliation
```markdown
Reconcile one analyzed chunk into an evidence-grounded final chunk record.
Inputs:
- segment manifest entry
- transcript slice
- chat slice
- carry-forward chat
- main analysis
- omissions analysis
- verification reports
- keyframe review
Rules:
1. Preserve original timestamps.
2. Remove malformed URLs.
3. Do not include any URL/tool/command unless supported by transcript, chat, carry-forward chat, local path, or keyframe review.
4. Do not claim a visual is visible unless keyframe review confirms it.
5. Preserve unanswered/deferred questions.
6. Preserve editorial warnings.
7. Deduplicate overlap by normalized timestamp + item type + text hash.
Return Markdown and JSON with:
- status: PASS | PASS_WITH_WARNINGS | FAIL
- counts
- timeline summary
- concepts
- Q&A
- links/tools/commands
- demos
- keyframes
- warnings
- student-material candidates
```
### Short Clip Analysis
```markdown
You are analyzing a short screen recording or app walkthrough.
Source:
- run_key: {{run_key}}
- recording_type: short_clip | bug_reproduction | design_feedback | app_walkthrough
- original_recording_range: 00:00:00 -> {{duration}}
- source_video_sha256: {{source_video_sha256}}
Hard rules:
1. Use timestamps relative to the clip unless an original recording timeline is provided.
2. Do not invent commands, files, UI labels, URLs, or implementation details.
3. Separate observed behavior from inferred intent.
4. Include timestamped screenshot/keyframe candidates for visual claims.
5. For a bug reproduction, produce repro steps and expected vs actual behavior.
6. For design feedback, produce requested changes and visual acceptance criteria.
7. For an app walkthrough, produce requirements and QA checks.
8. Include uncertainties and manual review points.
Return:
# Short Video Analysis: {{run_key}}
## 1. Summary
## 2. Observed Timeline
## 3. User Intent / Requirements
## 4. Reproduction Steps Or Walkthrough Steps
## 5. Expected Vs Actual Behavior
## 6. Implementation Notes
## 7. QA / Acceptance Criteria
## 8. Keyframes To Extract
## 9. Risks / Unknowns
## 10. Machine JSON
```
### Final Packet Generation
```markdown
Assemble a student follow-up packet from verified chunk records.
Inputs:
- segments/segment-manifest.v2.md
- chunks/*/final-reconciled.md
- chunks/*/final-reconciled.json
- chunks/*/verification/*.md
- final/keyframes-manifest.json if present
- resources extracted from chat/transcript
Rules:
1. Use original recording timestamps.
2. Do not include unverified visual claims.
3. Do not include malformed URLs.
4. Do not present experimental tools as official recommendations unless approved.
5. Keep policy-sensitive automation/cookie/quota/rate-limit sections as editorial notes.
6. Distinguish publication-ready files from reference drafts.
7. Deduplicate overlap and carry-forward repeated material.
Create:
- final/README.md
- final/student-followup-guide.md
- final/chapter-index.md
- final/resources-and-links.md
- final/qa-index.md
- final/exercises.md
- final/keyframes-manifest.md
- final/keyframes-manifest.json
- final/instructor-review-checklist.md
```
### Oracle Review
```markdown
[{{slug}}]
Project briefing:
{{project_briefing}}
Goal:
{{goal}}
Prior-State Ledger:
- prior_oracle_session: {{prior_oracle_session_or_none}}
- last_oracle_recommendation: {{summary_or_none}}
- implemented_since_prior_oracle: {{files_or_none}}
- verified_since_prior_oracle: {{commands_or_none}}
- failed_proof_or_blocker: {{receipt_or_none}}
- narrowing_decision: {{one_slice}}
- files_functions_now_in_scope: {{paths}}
- files_functions_out_of_scope: {{paths}}
- next_falsifiable_verification: {{command_or_receipt}}
- forward_progress_index: {{score_and_arithmetic}}
Deliverable:
Evaluate segmentation, chat carry-forward, analysis prompts, verification gates, final packet readiness, privacy/security risks, and the one highest-leverage next local action.
Return your answer as text in this response only. Do not create, attach, export, or offer any downloadable file. Do not create local project artifacts yourself. The local agent will write any needed files, plans, notes, goals, commits, or verification logs using local tools.
```
## Verification Gates
Treat `~/.oracle/sessions/<slug>/output.log`, browser logs, and raw Gemini responses as
private evidence. The MCP return can be truncated, so recover the full response from
the persisted session log before reconciliation. Before copying any material into
`publication/public-draft/` or another public wiki, scan and redact credentials,
auth/callback URLs, account/model identifiers, local usernames and paths,
signed/private URLs, notifications, and personal data. Gemini can repeat these even
when told not to.
Every pull quote and timestamp must be checked against local audio/captions and the
source frame or clip. Gemini-supplied quote timestamps are candidates, not receipts.
Record the verified timestamp and evidence path in the reconciliation artifact.
Run malformed URL checks on every final packet:
```bash
rg -n 'https://www\.\.\.|https://www\s|/"|\| .*: \||Context:' "$RUN_DIR/final" || true
```
Extract exact URLs from chat/transcript and compare against final resources:
```bash
rg -o 'https?://[^[:space:])>"'"'"']+' "$RUN_DIR/chunks" "$RUN_DIR/break-chat" \
| sed 's/[,:;.!"]$//' \
| sort -u \
> "$RUN_DIR/verification.expected-urls.txt"
rg -o 'https?://[^[:space:])>"'"'"']+' "$RUN_DIR/final/resources-and-links.md" \
| sed 's/[,:;.!"]$//' \
| sort -u \
> "$RUN_DIR/verification.final-urls.txt"
comm -23 "$RUN_DIR/verification.expected-urls.txt" "$RUN_DIR/verification.final-urls.txt" \
> "$RUN_DIR/verification.missing-urls.txt"
```
Validate timestamps stay within range, except carry-forward chat:
```bash
python3 scripts/validate-timestamps.py "$RUN_DIR" \
> "$RUN_DIR/receipts/timestamp-validation.log"
```
If the script does not exist yet, perform a manual timestamp pass and save it as `chunks/<nn>/verification/timestamp-report.md`.
Check machine JSON:
```bash
find "$RUN_DIR/chunks" -name '*.json' -print0 | xargs -0 -n1 jq empty
```
Required chunk gate files:
```text
chunks/<nn>/verification/coverage-report.md
chunks/<nn>/verification/hallucination-report.md
chunks/<nn>/verification/timestamp-report.md
chunks/<nn>/verification/qa-report.md
chunks/<nn>/final-reconciled.md
chunks/<nn>/final-reconciled.json
```
Chunk status meanings:
- `PASS`: complete and publication-safe after ordinary instructor review.
- `PASS_WITH_WARNINGS`: usable evidence, but warnings must be reviewed before publication.
- `FAIL`: rerun analysis, omissions, or reconciliation before using the chunk.
## Keyframes
Candidate extraction prompt output should include timestamp, reason, expected visual, priority, and confidence.
Extract deterministic frames:
```bash
mkdir -p "$RUN_DIR/chunks/01/keyframes/extracted"
ffmpeg -hide_banner -ss 00:23:48 \
-i "$RUN_DIR/inputs/source-video.mp4" \
-frames:v 1 \
"$RUN_DIR/chunks/01/keyframes/extracted/00-23-48-automation-verification.jpg"
```
Create contact sheet:
```bash
find "$RUN_DIR/chunks/01/keyframes/extracted" -name '*.jpg' | sort \
> "$RUN_DIR/chunks/01/keyframes/contact-sheet-inputs.txt"
ffmpeg -hide_banner -f concat -safe 0 \
-i <(sed "s#^#file '#; s#$#'#" "$RUN_DIR/chunks/01/keyframes/contact-sheet-inputs.txt") \
-vf "scale=320:-1,tile=3x4" \
"$RUN_DIR/chunks/01/keyframes/contact-sheet.jpg"
```
If process substitution is unavailable, generate the concat list as a real file and pass it to ffmpeg.
## Git Workflow
Initialize a new artifact project:
```bash
mkdir -p ~/dev/new-video-followup-project
cd ~/dev/new-video-followup-project
git init
cat > README.md <<'EOF'
# Video Follow-Up Pipeline
Local artifact project for repeatable video follow-up runs.
EOF
git add README.md
git commit -m "Initialize video follow-up project"
```
Use proof-oriented commit boundaries:
```bash
git commit -m "Capture source evidence for <run_id>"
git commit -m "Draft initial segmentation for <run_id>"
git commit -m "Record Oracle segmentation review for <run_id>"
git commit -m "Revise semantic chunk manifest"
git commit -m "Cut video chunks and ffprobe receipts"
git commit -m "Slice transcript and chat by chunk"
git commit -m "Run Gemini analysis for chunks 01-03"
git commit -m "Add verification reports for chunks 01-03"
git commit -m "Run omissions pass for chunk 01"
git commit -m "Reconcile verified chunk analyses"
git commit -m "Extract and review keyframe candidates"
git commit -m "Assemble student follow-up packet"
git commit -m "Document repeatable video follow-up process"
```
Default binary policy for future runs:
- Commit text artifacts, manifests, prompts, logs, JSON, Markdown, reports, and checksums.
- Do not commit large raw videos unless explicitly approved.
- Commit chunk videos only for local archival projects or when Git LFS is configured.
- Commit keyframes/contact sheets only when they are small enough and useful for review.
Suggested `.gitignore` for future automation projects:
```gitignore
runs/**/inputs/*.mp4
runs/**/inputs/*.mov
runs/**/inputs/*.mkv
runs/**/chunks/**/*.mp4
runs/**/chunks/**/*.mov
!runs/**/inputs/*.sha256
!runs/**/receipts/**
!runs/**/segments/**
!runs/**/chunks/**/*.json
!runs/**/chunks/**/*.md
!runs/**/chunks/**/*.log
!runs/**/final/**
!runs/**/oracle/**
runs/**/chunks/**/keyframes/extracted/*.jpg
runs/**/chunks/**/keyframes/extracted/*.png
runs/**/chunks/**/keyframes/strips/*.jpg
runs/**/chunks/**/keyframes/contact-sheet*.jpg
```
If committing binaries intentionally:
```bash
git add -f "$RUN_DIR/chunks/01/chunk.mp4"
git add -f "$RUN_DIR/chunks/01/keyframes/contact-sheet.jpg"
```
## Oracle Session Receipts
For every `$oracle` call, save:
```text
- slug
- date
- run id
- browser mode: true
- ChatGPT conversation URL
- bundle path
- bundle sha256
- prompt path
- output log path
- local decisions accepted/rejected
- commit before
- commit after
```
Template:
```markdown
# Oracle Session: {{session_slug}}
- date:
- local_run:
- browser_mode: true
- session_url:
- bundle_path:
- bundle_sha256:
- prompt_file:
- output_file:
- commit_before:
- commit_after:
## Question
## Oracle Response Summary
## Local Decisions
Accepted:
Rejected:
Follow-up:
```
## Final Packet Review
Treat these as first publication candidates:
- `final/README.md`
- `final/student-followup-guide.md`
- `final/chapter-index.md`
- `final/resources-and-links.md`
- `final/instructor-review-checklist.md`
Treat these as reference drafts until edited:
- `final/qa-index.md`
- `final/exercises.md`
- `final/keyframes-manifest.md`
Manual approval is mandatory for:
- platform automation
- browser automation
- cookies
- session replay
- quota/rate-limit behavior
- account credentials
- private emails
- private chat exports
- meeting transcript bot URLs
- student names
- screenshots of private repos
- local network addresses
- QR codes
- API keys
- package security claims
- third-party service recommendations
- public publishing
## End-To-End Checklist
Project setup:
- [ ] Create run folder.
- [ ] Copy source video into `inputs/`.
- [ ] Copy transcript/chat/screenshots/clips if available.
- [ ] Save source SHA256.
- [ ] Save ffprobe source receipt.
- [ ] Save tool versions.
- [ ] Commit source evidence receipts.
Long recording preprocessing:
- [ ] Run `silencedetect`.
- [ ] Review transcript and chat around silence spans.
- [ ] Draft boundary candidates.
- [ ] Create `segment-manifest.v1.json` and `.md`.
- [ ] Bundle planning context with `$oracle`.
- [ ] Run Oracle review in browser mode.
- [ ] Save Oracle session link/output locally.
- [ ] Create `segment-manifest.v2.json` and `.md`.
- [ ] Commit segmentation.
Chunk creation:
- [ ] Define canonical semantic chapters separately from <=4 minute technical analysis shards and map each shard back to a chapter.
- [ ] Re-encode Gemini analysis shards from the original as H.264 Main/yuv420p 29.97/30 fps + AAC-LC 44.1/48 kHz + faststart; do not use stream-copy or 4 fps/24 kHz mono analysis media.
- [ ] Keep each analysis shard below the configured attachment ceiling (target <=9 MiB when the ceiling is 10 MiB).
- [ ] `ffprobe` every chunk.
- [ ] Create `chunks/video-manifest.json`.
- [ ] Slice transcript per chunk.
- [ ] Slice chat per chunk.
- [ ] Preserve break chat separately.
- [ ] Add carry-forward chat to next spoken chunk.
- [ ] Add next-chunk preview.
- [ ] Commit chunk receipts and slices.
Chunk analysis:
- [ ] Pass both source-specific probes before batching: 30-60 second source-native sample and one production-size <=4 minute shard, with locally verified audio and visual facts.
- [ ] Create Gemini prompt per chunk.
- [ ] Attach MP4, prompt, transcript, chat, carry-forward chat, and preview.
- [ ] Save Gemini raw result/log.
- [ ] Verify live DOM submission (`user-query`, retained attachment, prompt absent from composer, begun/completed `model-response`) and an alphabetic-ending `END_OF_VIDEO_ANALYSIS` sentinel; wrapper logs alone are not proof.
- [ ] For long waits, prove `model-response message-content` growth; classify a no-growth placeholder over 2-3 snapshots/~3 minutes as stalled, re-encode, and retry once.
- [ ] Extract all required information from the response into local artifacts.
- [ ] Close the completed Gemini tab only after extraction and sentinel verification;
leave active Gemini tabs and the browser profile alive.
- [ ] Extract machine JSON.
- [ ] Build `expected-items.json`.
- [ ] Run coverage verification.
- [ ] Run hallucination/timestamp/Q&A checks.
- [ ] Run omissions pass if any gate fails.
- [ ] Commit analysis and verification reports.
Visual evidence:
- [ ] Generate keyframe candidates.
- [ ] Extract deterministic frames.
- [ ] Extract Gemini-proposed frames.
- [ ] Extract strips for visual demos.
- [ ] Create contact sheet.
- [ ] Review frames.
- [ ] Downgrade unsupported visual claims.
- [ ] Commit visual manifests/reviews.
Reconciliation:
- [ ] Merge main analysis, omissions, verification, and keyframe review.
- [ ] Write `final-reconciled.md/json` per chunk.
- [ ] Preserve warnings.
- [ ] Remove malformed URLs.
- [ ] Verify every pull quote and timestamp against local audio/captions and source frames.
- [ ] Redact credentials, auth/callback URLs, account/model identifiers, local usernames/paths, signed/private URLs, notifications, and personal data before publication; keep raw logs private.
- [ ] Deduplicate carried-forward overlap.
- [ ] Commit reconciled chunks.
Final packet:
- [ ] Generate final README.
- [ ] Generate student guide.
- [ ] Generate chapter index.
- [ ] Generate resources and links.
- [ ] Generate Q&A index.
- [ ] Generate exercises.
- [ ] Generate keyframes manifest.
- [ ] Generate instructor checklist.
- [ ] Run malformed URL scan.
- [ ] Run URL resolution check if public.
- [ ] Run public privacy/redaction scan and save `publication/private-redactions.md`.
- [ ] Run final packet review.
- [ ] Commit final packet.
Process documentation:
- [ ] Save this process as `docs/repeatable-video-followup-process.md`.
- [ ] Inspect for exact commands, mode branches, prompts, and gates.
- [ ] Commit process doc.
## Later Automation
Do not build these until the process doc is reviewed:
```text
scripts/video-followup/init-run.ts
scripts/video-followup/probe-source.ts
scripts/video-followup/detect-silence.ts
scripts/video-followup/build-segment-manifest.ts
scripts/video-followup/cut-chunks.ts
scripts/video-followup/slice-transcript.ts
scripts/video-followup/slice-chat.ts
scripts/video-followup/build-gemini-prompts.ts
scripts/video-followup/extract-machine-json.ts
scripts/video-followup/verify-coverage.ts
scripts/video-followup/scan-malformed-urls.ts
scripts/video-followup/extract-keyframes.ts
scripts/video-followup/build-contact-sheet.ts
scripts/video-followup/reconcile-chunk.ts
scripts/video-followup/build-final-packet.ts
templates/gemini/chunk-analysis.md
templates/gemini/omissions.md
templates/oracle/planning-review.md
templates/oracle/final-review.md
templates/final/README.md
templates/final/student-followup-guide.md
templates/final/chapter-index.md
templates/final/resources-and-links.md
templates/final/instructor-review-checklist.md
docs/automation-roadmap.md
```
Automation priority:
1. Malformed URL scanner.
2. Timestamp validator.
3. Expected-items builder from chat/transcript.
4. Chunk prompt generator.
5. ffmpeg chunk cutter.
6. Keyframe extractor/contact sheet builder.
7. Final packet assembler.