Abby teaches Trustworthy Selling, in Chinese

The source course, served per learner

Trustworthy Selling is an English course of 103 lesson videos. The master lives on the studio's render machine, the serving copies in a SharePoint library, and the course site plays each video through a per-user link so SharePoint's own permissions apply. The Chinese edition is designed to sit on the same lesson page, beside the English original, not to replace it.

trustworthy.apexfamilywealth.com · SharePoint KnowledgeBase / Trustworthy Selling / Mod N · scripts/upload-videos.mjs (resumable, 10 MiB chunks) · public/lessons/videos.json

Transcribe with word timings

Whisper runs on the render machine and writes one transcript per video with text and word timings. The English sentence is kept all the way through the pipeline: it becomes the ghost line behind each Chinese line on screen and the anchor the translation is aligned against.

content/transcripts/v0NN.json · manifest.tsv (id → module, file, duration)

Translate under a glossary, gate every lesson by alignment

Translation agents work from a fixed prompt and a course glossary so the course's terms are rendered the same way in every lesson. One lesson was translated blind first and compared sentence by sentence before the bulk run went to nine agents in parallel. Every lesson passes a parse-and-alignment check, and the output is a segmented file of English and Chinese pairs, which is the pipeline's real input.

scripts/translate/PROMPT.md · GLOSSARY.md · content/transcripts/v0NN.zh.json

Voice each segment; the timings file is the single clock

A narration tool sends each Chinese segment to Fish Audio in Abby's voice, joins the clips with a 0.35 s gap, adds the title and end card pauses, and writes one WAV plus a timings file. The composition reads that same file for both audio offsets and clip offsets, so picture and sound cannot drift apart. When no API key is present the tool falls back to a local voice so the pipeline can still be exercised end to end.

build/tts.py · assets/<id>/narration.wav + timings.json · Fish model s2.1-pro-free in development · fallback Kokoro zf_xiaobei

The avatar follows our audio, never a script

Abby is a HeyGen photo avatar Tony created. The video request sends the narration as an uploaded audio asset rather than a script, so HeyGen only moves the lips and the Fish narration stays byte-identical to what the composition already uses. The two engines were compared on the same 13.7 s slice of real lesson audio, same look, same 1080p, one after the other so each credit delta belongs to one job.

Two traps written into the pipeline notes: for a photo avatar the look id equals the group id, which the CLI does not say; and the free tier watermarks every clip and caps Avatar IV at one minute, so the production runs were made on Pro. Lip-sync quality itself is only judgeable by watching, and that is Tony's call.

heygen asset create → audio_asset_id · heygen video create -d request.json --wait · videos/heygen-abby-test/run_test.py + results.json

Measure the returned clip against the upload, every time

The audio inside a returned HeyGen clip is not sample-aligned with the WAV that was uploaded. One job led by 13 ms, another lagged by 23 ms, and the sign differed. The build cross-correlates the clip's audio with the upload, writes the offset to a file, and starts the voice track that much earlier or later, so the lips and the voice the viewer hears agree within a few milliseconds.

assets/abby.offset.json · AUDIO_LEAD read by the build script · final voice vs lip-driving audio within 0–8 ms on the measured jobs

Composite in HyperFrames: a notebook page with a presenter zone

One lesson template is instantiated per video from the timings file. The look matches the course site: navy canvas, gold underline markers on terms, Noto Sans SC, the English sentence as low-opacity ghost type behind each Chinese line, a segment counter and a hairline progress rule so a twelve-minute lesson never feels unanchored. Abby sits in a fixed zone and the caption strip is kept clear of it, so text never runs under the presenter.

compositions/lesson.template.html · build/compose.mjs · build/proof.mjs · navy #001d69 · gold #F7E14D · paper #fbfaf8

Check, render, route by size, register

One runner does a whole batch: narrate if the timings are missing or the provider changed, compose, run the HyperFrames check and refuse to render unless it passes, render at 720p streaming frames straight to the encoder, then route. A render under 25 MiB is copied into the site for Pages; a larger one waits for the SharePoint uploader. Either way the lesson's entry in the video index gains a Chinese edition with its size, duration and voice provider. The runner refuses to start without disk headroom, because a full disk once wedged every shell on the machine.

build/run.mjs v001 … v010 · npx hyperframes check . · render --quality looks --crf 24 · public/media/zh/<id>.mp4 · videos.json[mod][file].zh

Review loops, and the redesign they forced

Pronunciation review, voice review and module reviews each have a folder in the project. Tony judged the first ten editions "subtitles of the narration", and he was right: text that only restates the voice is not an animation. The next version moved to a scene grammar, a diagram that builds, a facet that comes into focus, a count-up, growth bars, a meter, chips and a lockup, with a storyboard per video and the narration demoted to a caption strip. The avatar question followed from that review, not before it.

pronunciation-review/ · voice-review/ · module1-review/ · module2-review/ · STORYBOARD.md · design-sample/ · source-led/