encoding WebM and notes on irrsi

Sunday June 14, 2015

For the first time I started encoding videos in WebM (replacing GIF), I had fun learning something new. The learning curve, at least to me, was not steep because I already had prior experience using the ffmpeg to encode H.264 videos. This entry on Haven.NightlyArt will serve as a note on what problems that challenged me throughout the process, and reminders for those who found this article. I assume my readers here already have basic experience with ffmpeg, hence this will not serve as a tutorial for newcomers.

notes on browser compatibility

I should’ve checked this first.

WebM container support

I was compiling the videos with libvpx-vp9 codec, which is not widely supported yet, for instance Apple mobile devices. I thought the Safari mobile on iOS 8 does not support the VP9 codec yet, but after a quick confirmation it is the WebM container itself that is not supported by the browser. So if you are targeting mobile users, it is a better preference to use the H.264/MP4 format.

H.264 format support

encoding strategy

The whole idea was to replace GIF, because GIF has a very low animation quality and it is an old format, belonging to the BBS era (that’s certainly an exaggeration). For your information, WebP is an image format (currently being developed by Google) to replace PNG, JPEG, and GIF. As you might guess, it hasn’t gained widespread adoption yet on the browser market, thus not a good choice to replace GIF either.

Most of the input videos that I used while conducting the experiment were H.264 1280x720 HD videos, all of them from anime (Food Wars, Attack on Titan, Sword Art Online 2, and Shakugan No Shana S3). How did I get those videos? Not telling hahaha.

To embed a full 1280x720 is not a good idea. 720x405 is good enough for a short animation, and let’s have mercy on mobile users with mobile broadband connection. That aim was to produce a small animated clip with lower file size while not compromising the quality. Ideally, the audio stream must not get included because (1) it is annoying to have an autoplay, on-loop video with audio stream, (2) a bit smaller file size.

encoding process

A typical anime is about 24 minutes long, and of course we are not going to include the whole shebang. For trimming purpose, we will use ffmpeg

ffmpeg -i input_video.mp4 -ss hh:mm:ss -t hh:mm:ss -c copy output_video.mp4

-c copy option means that we want to retain the same format for all streams (video, audio, and subtitle if available). If the input is from MP4 container, it is easy. But things get awry if the input is from MKV container, because that’s where most problems creep in.

not in sync

I tried to trim episode 21 Attack on Titan. When the trim was done, the audio was not in-sync with the video but the subtitle was in sync. This situation didn’t really bother me that much because I didn’t need audio stream, but it called for a concern: how do we get the audio to be in-sync with the video?

When dealing with MKV file, usually I will use the option -c:s mov_text to change the subtitle format from ASS to MOV_TEXT. This is because I want the output video in MP4 container format, and MP4 container format only supports mov_text subtitle format. For MKV, it supports ASS, SRT, and SSA.

ffmpeg -i input_video.mkv -ss hh:mm:ss -t hh:mm:ss -c:a copy -c:v copy -c:s mov_text output_video.mp4

As mentioned before, ffmpeg could not get the encode done right. The audio stream was off by few seconds delay, even after I changed the audio codec to something else like libfdk_aac. I tried to encode the video with mencoder, and this is what happened.

mencoder -ss hh:mm:ss -endpos hh:mm:ss -oac copy -ovc copy input_video.mkv -o output_video.mp4

-oac copy and -ovc copy options tell the mencoder to retain audio and video stream format. After running that mencoder command above, it threw me an error (fully duplicated below).

MEncoder 1.1-4.2.1 (C) 2000-2012 MPlayer Team

WARNING: OUTPUT FILE FORMAT IS _AVI_. See -of help.
success: format: 0  data: 0x0 - 0x244b21e6
libavformat version 54.6.100 (internal)
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (aac), -aid 0, -alang jpn
[lavf] stream 2: subtitle (movtext), -sid 0, -slang eng
[lavf] stream 3: subtitle (movtext), -sid 1, -slang eng
VIDEO:  [H264]  1280x720  24bpp  23.976 fps  3164.1 kbps (386.2 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1280x720  fps:23.976  ftime:=0.0417
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
libavcodec version 54.23.100 (internal)
AUDIO: 48000 Hz, 2 ch, s16le, 188.6 kbit/12.28% (ratio: 23578->192000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
videocodec: framecopy (1280x720 24bpp fourcc=34363248)
Audio format 0x4134504d is incompatible with '-oac copy', please try '-oac pcm' instead or use '-fafmttag' to override it.

Exiting..

Bingo! We found the problem!

mencoder tells us here that -oac copy isn’t compatible with the audio stream, suggesting us to use -oac pcm instead. I ran the same command with new option -oac pcm, and it worked! The audio stream was in-sync.

There was a but.

The subtitle stream wasn’t in there anymore. I tried using the -s option to include the subtitle, but it didn’t work. To extract subtitle from a video container, use mkvextract.

mkvextract tracks input_video.mkv <track_number>:<output_file.ext>
mkvextract tracks input_video.mkv 2:subtitle.ass

Or maybe mencoder doesn’t understand ASS subtitle?

I installed ass2srt package (via Gem), converted the ASS subtitle to SRT, and tried inserting it as a stream into the video container with mencoder.

sudo gem install ass2srt
ass2srt subtitle.ass

mencoder -ss hh:mm:ss -endpos hh:mm:ss -oac copy -ovc copy input_video.mkv -s subtitle.srt -o output_video.mp4

It didn’t work either. The next viable option was to hardcode the subtitle into the video stream file with ffmpeg.

ffmpeg -i input_video.mp4 -c:a libfdk_aac -vf subtitles=subtitle.srt output_video_hardcodedsub.mp4

The problem with hardcoding the subtitle into the video stream is that ffmpeg will re-encode the whole video stream. The -vf option is mutually exclusive with -c:v option, thus it cannot be used together. Since we will re-encode the video stream, expect a very long time to complete the process.

My 2 cents: avoid re-encoding MKV to MP4 if possible.

encoding to WebM

ffmpeg -i input_trimmed.mp4 -c:v libvpx-vp9 -b:v 500k -threads 4 -speed 2 -tile-columns 6 -frame-parallel 1 -auto-alt-ref 1 -lag-in-frames 25 -s 720x405 -an final.webm

The command above is the modified version from this guide: VP9 Encoding Guide. -s 720x405 scales the video from original input resolution down to 720x405 (maintained aspect ratio from 1280x720). The original guide instructs you to do a 2-pass, but ain’t nobody got time to do a 2-pass.

With H.264 video format

ffmpeg -i input_trimmed.mp4 -c:v libx264 -profile:v high -preset slow -s 720x406 -crf 30 -an final.mp4

ffmpeg -i input_trimmed.mp4 -c:v libx264 -profile:v high -preset slow -s 720x406 -an final.mp4

By manipulating the -crf (constant rate factor, the lower the better), we can manipulate the output file size. Usually a value between 18 and 24 is chosen, but because we intend to produce a video file with a much less size, -crf 30 is a good choice.

Important Note! There is a limitation regarding scaling with libx264. We cannot down-scale the video resolution with odd number when using the libx264 encode library, as explained here on Stack Overflow by Andy Hin.

encoding results

The fun part!

the file size

WebM VP9          = 1.1 MB
MP4 H.264         = 1.7 MB
MP4 H.264 CRF 30  = 699 KB

Video with WebM container, VP9 video stream format.

Video with MP4 container, H264 video stream format, CRF 30.

Note You might notice the lag. I don’t know why, and too lazy to inspect.

notes on irssi

brew install irssi
vim .irssi/config

irssi
/server rizon

/msg NickServ HELP
/window #
IDENTIFY <password>
/window #

/join #channel
/msg <dcc-bot> xdcc send <package ID>
/dcc get <bot name>
/leave

/disconnect
/quit

A brief overview of my irssi workflow. The irssi documentation has it all.