For the first time I started encoding videos in WebM (replacing GIF), I had fun learning something new. The learning curve, at least to me, was not steep because I already had prior experience using the ffmpeg to encode H.264 videos. This entry on Haven.NightlyArt will serve as a note on what problems that challenged me throughout the process, and reminders for those who found this article. I assume my readers here already have basic experience with ffmpeg, hence this will not serve as a tutorial for newcomers.
notes on browser compatibility
I should’ve checked this first.
I was compiling the videos with libvpx-vp9 codec, which is not widely supported yet, for instance Apple mobile devices. I thought the Safari mobile on iOS 8 does not support the VP9 codec yet, but after a quick confirmation it is the WebM container itself that is not supported by the browser. So if you are targeting mobile users, it is a better preference to use the H.264/MP4 format.
encoding strategy
The whole idea was to replace GIF, because GIF has a very low animation quality and it is an old format, belonging to the BBS era (that’s certainly an exaggeration). For your information, WebP is an image format (currently being developed by Google) to replace PNG, JPEG, and GIF. As you might guess, it hasn’t gained widespread adoption yet on the browser market, thus not a good choice to replace GIF either.
Most of the input videos that I used while conducting the experiment were H.264 1280x720 HD videos, all of them from anime (Food Wars, Attack on Titan, Sword Art Online 2, and Shakugan No Shana S3). How did I get those videos? Not telling hahaha.
To embed a full 1280x720 is not a good idea. 720x405 is good enough for a short animation, and let’s have mercy on mobile users with mobile broadband connection. That aim was to produce a small animated clip with lower file size while not compromising the quality. Ideally, the audio stream must not get included because (1) it is annoying to have an autoplay, on-loop video with audio stream, (2) a bit smaller file size.
encoding process
A typical anime is about 24 minutes long, and of course we are not going to include the whole shebang. For trimming purpose, we will use ffmpeg
ffmpeg -i input_video.mp4 -ss hh:mm:ss -t hh:mm:ss -c copy output_video.mp4
-c copy option means that we want to retain the same format for all streams (video, audio, and subtitle if available). If the input is from MP4 container, it is easy. But things get awry if the input is from MKV container, because that’s where most problems creep in.
not in sync
I tried to trim episode 21 Attack on Titan. When the trim was done, the audio was not in-sync with the video but the subtitle was in sync. This situation didn’t really bother me that much because I didn’t need audio stream, but it called for a concern: how do we get the audio to be in-sync with the video?
When dealing with MKV file, usually I will use the option -c:s mov_text to change the subtitle format from ASS to MOV_TEXT. This is because I want the output video in MP4 container format, and MP4 container format only supports mov_text subtitle format. For MKV, it supports ASS, SRT, and SSA.
ffmpeg -i input_video.mkv -ss hh:mm:ss -t hh:mm:ss -c:a copy -c:v copy -c:s mov_text output_video.mp4
As mentioned before, ffmpeg could not get the encode done right. The audio stream was off by few seconds delay, even after I changed the audio codec to something else like libfdk_aac. I tried to encode the video with mencoder, and this is what happened.
mencoder -ss hh:mm:ss -endpos hh:mm:ss -oac copy -ovc copy input_video.mkv -o output_video.mp4
-oac copy and -ovc copy options tell the mencoder to retain audio and video stream format. After running that mencoder command above, it threw me an error (fully duplicated below).
MEncoder 1.1-4.2.1 (C) 2000-2012 MPlayer Team
WARNING: OUTPUT FILE FORMAT IS _AVI_. See -of help.
success: format: 0 data: 0x0 - 0x244b21e6
libavformat version 54.6.100 (internal)
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (aac), -aid 0, -alang jpn
[lavf] stream 2: subtitle (movtext), -sid 0, -slang eng
[lavf] stream 3: subtitle (movtext), -sid 1, -slang eng
VIDEO: [H264] 1280x720 24bpp 23.976 fps 3164.1 kbps (386.2 kbyte/s)
[V] filefmt:44 fourcc:0x34363248 size:1280x720 fps:23.976 ftime:=0.0417
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
libavcodec version 54.23.100 (internal)
AUDIO: 48000 Hz, 2 ch, s16le, 188.6 kbit/12.28% (ratio: 23578->192000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
videocodec: framecopy (1280x720 24bpp fourcc=34363248)
Audio format 0x4134504d is incompatible with '-oac copy', please try '-oac pcm' instead or use '-fafmttag' to override it.
Exiting..
Bingo! We found the problem!
mencoder tells us here that -oac copy isn’t compatible with the audio stream, suggesting us to use -oac pcm instead. I ran the same command with new option -oac pcm, and it worked! The audio stream was in-sync.
There was a but.
The subtitle stream wasn’t in there anymore. I tried using the -s option to include the subtitle, but it didn’t work. To extract subtitle from a video container, use mkvextract.
mkvextract tracks input_video.mkv <track_number>:<output_file.ext>
mkvextract tracks input_video.mkv 2:subtitle.ass
Or maybe mencoder doesn’t understand ASS subtitle?
I installed ass2srt package (via Gem), converted the ASS subtitle to SRT, and tried inserting it as a stream into the video container with mencoder.
sudo gem install ass2srt
ass2srt subtitle.ass
mencoder -ss hh:mm:ss -endpos hh:mm:ss -oac copy -ovc copy input_video.mkv -s subtitle.srt -o output_video.mp4
It didn’t work either. The next viable option was to hardcode the subtitle into the video stream file with ffmpeg.
ffmpeg -i input_video.mp4 -c:a libfdk_aac -vf subtitles=subtitle.srt output_video_hardcodedsub.mp4
The problem with hardcoding the subtitle into the video stream is that ffmpeg will re-encode the whole video stream. The -vf option is mutually exclusive with -c:v option, thus it cannot be used together. Since we will re-encode the video stream, expect a very long time to complete the process.
My 2 cents: avoid re-encoding MKV to MP4 if possible.
encoding to WebM
ffmpeg -i input_trimmed.mp4 -c:v libvpx-vp9 -b:v 500k -threads 4 -speed 2 -tile-columns 6 -frame-parallel 1 -auto-alt-ref 1 -lag-in-frames 25 -s 720x405 -an final.webm
The command above is the modified version from this guide: VP9 Encoding Guide. -s 720x405 scales the video from original input resolution down to 720x405 (maintained aspect ratio from 1280x720). The original guide instructs you to do a 2-pass, but ain’t nobody got time to do a 2-pass.
With H.264 video format
ffmpeg -i input_trimmed.mp4 -c:v libx264 -profile:v high -preset slow -s 720x406 -crf 30 -an final.mp4
ffmpeg -i input_trimmed.mp4 -c:v libx264 -profile:v high -preset slow -s 720x406 -an final.mp4
By manipulating the -crf (constant rate factor, the lower the better), we can manipulate the output file size. Usually a value between 18 and 24 is chosen, but because we intend to produce a video file with a much less size, -crf 30 is a good choice.
Important Note! There is a limitation regarding scaling with libx264. We cannot down-scale the video resolution with odd number when using the libx264 encode library, as explained here on Stack Overflow by Andy Hin.
encoding results
The fun part!
WebM VP9 = 1.1 MB
MP4 H.264 = 1.7 MB
MP4 H.264 CRF 30 = 699 KB
Video with WebM container, VP9 video stream format.
Video with MP4 container, H264 video stream format, CRF 30.
Note You might notice the lag. I don’t know why, and too lazy to inspect.
notes on irssi
brew install irssi
vim .irssi/config
irssi
/server rizon
/msg NickServ HELP
/window #
IDENTIFY <password>
/window #
/join #channel
/msg <dcc-bot> xdcc send <package ID>
/dcc get <bot name>
/leave
/disconnect
/quit
A brief overview of my irssi workflow. The irssi documentation has it all.