Skip to content

Tutorials

Make a vertical short with captions

From a phone clip to a 9:16 video with word-highlighted captions, a colour filter and music that lowers under your voice, with the CLI.

Last updated:

You will learn#

  • how to set up a 9:16 project and place a clip;
  • how to caption the speech and style the captions;
  • how to add music that ducks under the voice;
  • how to check the result and export it.

What you need#

A video file with speech, talk.mp4, and a music file, song.mp3. Transcription runs on your computer with a model you install once (see Transcription). The identifiers below are examples: pass your own with --id and --asset-id as here, or use the ones the commands print.

Steps#

  1. Create the project. The default preset is 9:16 at 30 frames per second, which is the format of a short.

    Terminal
    vukem-video project:create --name "Short 01" --id prj_7k2m9x4q1b
  2. Import the two files and wait until they are ready.

    Terminal
    vukem-video media:import talk.mp4 --project prj_7k2m9x4q1b --asset-id ast_3c5v8n2w6d
    vukem-video media:import song.mp3 --project prj_7k2m9x4q1b --asset-id ast_4h6j8k2m3n
  3. Add three tracks (footage, captions, music) and put the clip and the song on them. A track has a kind, visual or audio.

    Terminal
    vukem-video edit --project prj_7k2m9x4q1b --ops - <<'JSON'
    [{"op":"track.add","args":{"kind":"visual","name":"Video","id":"trk_f6g7h8j9k0"}},
     {"op":"track.add","args":{"kind":"visual","name":"Captions","id":"trk_a1b2c3d4e5"}},
     {"op":"track.add","args":{"kind":"audio","name":"Music","id":"trk_m1n2p3q4r5"}},
     {"op":"clip.add","args":{"track_id":"trk_f6g7h8j9k0","asset_id":"ast_3c5v8n2w6d","start":"0s","id":"itm_m1n2p3q4r5"}},
     {"op":"audio.add","args":{"track_id":"trk_m1n2p3q4r5","asset_id":"ast_4h6j8k2m3n","start":"0s","id":"itm_s6t7v8w9x0"}}]
    JSON
  4. Transcribe the speech and make the captions from it. The first command downloads the model once and asks before it does.

    Terminal
    vukem-video tools:install whisper --model small
    vukem-video transcribe --project prj_7k2m9x4q1b --asset ast_3c5v8n2w6d --language en --wait
    vukem-video edit --project prj_7k2m9x4q1b caption.from_transcript \
      clip_item_id=itm_m1n2p3q4r5 track_id=trk_a1b2c3d4e5 max_words:=4
  5. Style the captions: big, upper case, with the spoken word highlighted.

    Terminal
    vukem-video edit --project prj_7k2m9x4q1b track.set_caption_style track_id=trk_a1b2c3d4e5 \
      style:='{"uppercase":true,"size":84,"highlight":"word","highlight_color":"#FFE600","position":"bottom"}'
  6. Give the footage a warmer look.

    Terminal
    vukem-video edit --project prj_7k2m9x4q1b clip.set_filter item_id=itm_m1n2p3q4r5 filter=warm amount:=0.5
  7. Lower the music while you speak. Wait until the song is ready first; ducking needs the analysis of the speech in the clip.

    Terminal
    vukem-video edit --project prj_7k2m9x4q1b audio.duck item_id=itm_s6t7v8w9x0 \
      speech_item_ids:='["itm_m1n2p3q4r5"]' gain_db:=-14
  8. Look at a frame, run the lint and export.

    Terminal
    vukem-video view --project prj_7k2m9x4q1b --at 2s
    vukem-video lint --project prj_7k2m9x4q1b
    vukem-video export --project prj_7k2m9x4q1b

The export prints the verified file under <workspace>/exports/prj_7k2m9x4q1b/.

Common mistakes#

  • Captions inside the unsafe margin of a social app: read The safe area and run the lint.
  • Ducking before the clip is analysed: E_INVALID_STATE; wait for the asset and send it again.
  • Running caption.from_transcript before the transcript exists. Wait for the transcribe job.

Next steps#