paazmaya.fi

The Website of Juga Paazmaya | Stories about Web Development, Japanese Martial Arts, Hardware prototyping and travelling

Making a video of CasperJS screen captures with FFmpeg

CasperJS is a browser automation tool that can be used for testing web sites without manual interaction. It can also take screen captures. The screen captures most likely vary between different browsers and their versions, which makes it reasonable to use all the possible browser options there is. At the moment CasperJS supports PhantomJS version 1.9 and SlimerJS version 0.9. In the near future there should also come the availability of Internet Explorer via TrifleJS, in its version 0.8. PhantomJS is based on Webkit, which in is the base for most mobile device browsers and Google Chrome, while SlimerJS uses a recent version of Mozilla Firefox installed in the same environment.

FFmpeg is a command line tool for doing media conversion of nearly any format. It is used by most of the online video services, most notably Youtube, for their media conversion at the server side.

Below is shown an example initialisation of CasperJS, which receives listeners for step.start and step.complete events. When those events occur, a screen shot of the given state is taken and stored in the directory specified by the screensDir variable. The file names are made of equal length by filling leading zeroes on the counter. This is needed for the video conversion process.

var screensDir = 'captures/';
var stepCounter = 0;

var casper = require('casper').create({
  verbose: true,
  viewportSize: {
    width: 1200,
    height: 900
  }
});

var stepCapture = function () {
  var str = String(++stepCounter);
  while (str.length < 5) {
    str = '0' + str;
  }
  casper.capture(screensDir + 'step-complete-' + str + '.png', {
    top: 0,
    left: 0,
    width: 1200,
    height: 900
  });
};

casper.on('step.start', stepCapture);
casper.on('step.complete', stepCapture);

// Steps...

casper.run();

After all the steps have been executed and the screen captured are saved, the video conversion can be started. The frame rate of four will make each picture to be seen for 250 milliseconds and since there is a screen capture in both start and end of a given step, they most likely look quite the same. Therefore about half of a second will appear the same.

ffmpeg -framerate 4 -i 'captures/step-complete-%05d.png' \
  -pix_fmt yuv420p -c:v libx264 steps-completed.mp4

Once the video has been created, it should have the same resolution as used in the capture() method, 1200 pixels wide and 900 pixels high.

Subtitles!

How about adding subtitles for showing the URL of the page that is seen at a given time in the video?

SRT is pretty simple subtitle format and can be created as seen below in the stepSubtitles() function.

The incrementation of the stepCounter is done just before it is passed to the subtitles array, which needs the index to begin from 1, while the earlier parts of the calculations are based on 0 index.

Here what needs to be added on the previous example:

// Load file system module for writing subtitle file
var fs = require('fs');

var frameRate = 4;
var subtitles = [];

var fillZeroes = function (num, len) {
  len = len || 2;
  var str = String(num);
  while (str.length < len) {
    str = '0' + str;
  }
  return str;
};

var calculateSrtTime = function (seconds) {
  var h = Math.floor(seconds / 3600);
  var m = Math.floor((seconds - h * 60) / 60);
  var s = Math.floor(seconds - (h * 3600 + m * 60));
  var ms = Math.floor((seconds - (h * 60 + m * 60 + s)) * 1000);
  return fillZeroes(h) + ':' + fillZeroes(m) + ':' +
    fillZeroes(s) + ',' + fillZeroes(ms, 3);
};

var stepSubtitles = function () {
  var url = casper.getCurrentUrl();
  var seconds = stepCounter / frameRate;
  var times = calculateSrtTime(seconds - 1 / frameRate) + ' --> ' + calculateSrtTime(seconds);
  subtitles.push(
    ++stepCounter,
    times,
    url,
    ''
  );
};

casper.on('exit', function () {
  var data = subtitles.join('\n');
  var stream = fs.open(screensDir + 'steps-subtitles.srt', 'w');
  stream.write(data);
  stream.close();
});

Now the fillZeroes() function can be used to replace the first lines of stepCapture(). Also the calling of stepSubtitles() should be done inside the stepCapture in its last row.

Once that runs, there should now be along with the images, the subtitle file called steps-subtitles.srt. The video conversion command now looks like:

ffmpeg -framerate 4 -i 'captures/step-complete-%05d.png' \
  -sub_charenc ISO-8859-1 -i captures/steps-subtitles.srt \
  -scodec mov_text -metadata:s:s:0 language=eng \
  -pix_fmt yuv420p -c:v libx264 steps-completed-subtitled.mp4

As a result, ~83 megabytes of PNG images becomes ~2.1 megabyte video, while FFmpeg has chosen to use 340 Kbps video bitrate for the 1200x900 pixels resolution.

Output from Mediainfo:

Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom
File size                                : 2.11 MiB
Duration                                 : 51s 750ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 342 Kbps
Writing application                      : Lavf54.63.104

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.2
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 51s 750ms
Bit rate                                 : 340 Kbps
Width                                    : 1 200 pixels
Height                                   : 900 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Constant
Frame rate                               : 4.000 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.079
Stream size                              : 2.09 MiB (99%)
Writing library                          : x264 core 142 r2389 956c8d8

Text
ID                                       : 2
Format                                   : Timed Text
Muxing mode                              : sbtl
Codec ID                                 : tx3g
Duration                                 : 51s 750ms
Bit rate mode                            : Variable
Bit rate                                 : 348 bps
Stream size                              : 2.20 KiB (0%)
Language                                 : English

By adding -x264opts keyint=1 on the video conversion command would make every frame a key frame, making the images clearer and seeking faster, but at the expense of increasing the file size dramatically.

Fairly simple, right? Increasing the frame rate in the JavaScript variable and in the FFmpeg command, the captured images can be browsed through quickly and possible test reviews can be quick.

Any anomalies might be difficult to spot, but at least a general idea of what pages and places the steps are passing, can be figured out.