You will learn#
- how to set up a 9:16 project and place a clip;
- how to caption the speech and style the captions;
- how to add music that ducks under the voice;
- how to check the result and export it.
What you need#
A video file with speech, talk.mp4, and a music file, song.mp3. Transcription runs on your computer with a model you install once (see Transcription). The identifiers below are examples: pass your own with --id and --asset-id as here, or use the ones the commands print.
Steps#
-
Create the project. The default preset is 9:16 at 30 frames per second, which is the format of a short.
Terminal vukem-video project:create --name "Short 01" --id prj_7k2m9x4q1b -
Import the two files and wait until they are ready.
Terminal vukem-video media:import talk.mp4 --project prj_7k2m9x4q1b --asset-id ast_3c5v8n2w6d vukem-video media:import song.mp3 --project prj_7k2m9x4q1b --asset-id ast_4h6j8k2m3n -
Add three tracks (footage, captions, music) and put the clip and the song on them. A track has a kind,
visualoraudio.Terminal vukem-video edit --project prj_7k2m9x4q1b --ops - <<'JSON' [{"op":"track.add","args":{"kind":"visual","name":"Video","id":"trk_f6g7h8j9k0"}}, {"op":"track.add","args":{"kind":"visual","name":"Captions","id":"trk_a1b2c3d4e5"}}, {"op":"track.add","args":{"kind":"audio","name":"Music","id":"trk_m1n2p3q4r5"}}, {"op":"clip.add","args":{"track_id":"trk_f6g7h8j9k0","asset_id":"ast_3c5v8n2w6d","start":"0s","id":"itm_m1n2p3q4r5"}}, {"op":"audio.add","args":{"track_id":"trk_m1n2p3q4r5","asset_id":"ast_4h6j8k2m3n","start":"0s","id":"itm_s6t7v8w9x0"}}] JSON -
Transcribe the speech and make the captions from it. The first command downloads the model once and asks before it does.
Terminal vukem-video tools:install whisper --model small vukem-video transcribe --project prj_7k2m9x4q1b --asset ast_3c5v8n2w6d --language en --wait vukem-video edit --project prj_7k2m9x4q1b caption.from_transcript \ clip_item_id=itm_m1n2p3q4r5 track_id=trk_a1b2c3d4e5 max_words:=4 -
Style the captions: big, upper case, with the spoken word highlighted.
Terminal vukem-video edit --project prj_7k2m9x4q1b track.set_caption_style track_id=trk_a1b2c3d4e5 \ style:='{"uppercase":true,"size":84,"highlight":"word","highlight_color":"#FFE600","position":"bottom"}' -
Give the footage a warmer look.
Terminal vukem-video edit --project prj_7k2m9x4q1b clip.set_filter item_id=itm_m1n2p3q4r5 filter=warm amount:=0.5 -
Lower the music while you speak. Wait until the song is ready first; ducking needs the analysis of the speech in the clip.
Terminal vukem-video edit --project prj_7k2m9x4q1b audio.duck item_id=itm_s6t7v8w9x0 \ speech_item_ids:='["itm_m1n2p3q4r5"]' gain_db:=-14 -
Look at a frame, run the lint and export.
Terminal vukem-video view --project prj_7k2m9x4q1b --at 2s vukem-video lint --project prj_7k2m9x4q1b vukem-video export --project prj_7k2m9x4q1b
The export prints the verified file under <workspace>/exports/prj_7k2m9x4q1b/.
Common mistakes#
- Captions inside the unsafe margin of a social app: read The safe area and run the lint.
- Ducking before the clip is analysed:
E_INVALID_STATE; wait for the asset and send it again. - Running
caption.from_transcriptbefore the transcript exists. Wait for thetranscribejob.