An important part of the experimentation process is having the right tools. The right tools allow us to measure the right data, detect trends over time, and make informed decisions throughout our analyses. Early in the development of Cairo it became clear that I would need the ability to visually analyze the codec at runtime to both ensure correctness and look for areas for improvement. As a result, I created a simple utility known plainly as Codec Tool.
Codec Tool allows the user to load an image, video file, or a live desktop capture, and then process the content using Cairo and analyze the results.
The left video view in this screenshot shows the current source frame, which in this case is an h.264 encoded 1280x720 frame. The center image is togglable, and currently shows the state of Cairo's most recent prediction frame. The rightmost image is the Cairo output of the source frame. Note that while our source is in a compressed format for this example, in practice only raw content formats are used for experimentation.
The center image can be changed to examine the internal state of Cairo. This includes views of the source, color converted source, all prediction frames, motion block data, transformed frame, quantized frame, and more.
In the screenshot above we examine the motion block table for the current frame. Each block is color coded by type to indicate the form of motion compensation. For example, green blocks indicate a skip, teal indicates a motion-skip, and blue indicates a general motion predicted block.
Codec Tool wasn't designed to be a general purpose image and video viewer, so it lacks many common features such as audio playback. It does however have a number of features
that are useful to codec developers:
Load any video or image format supported by FFMpeg and play, pause, and skip around the content. Save out individual frames or sections of the Cairo encoded output. Inspect individual blocks or pixels to ensure they have the correct value.
Zoom and Filter
Codec Tool is built on Imagine, so you can zoom in to inspect a detail, or apply an array of filters to distort the content in particular ways.
Encode Order Control
Re-order, repeat, or skip frames as they are fed into Cairo to quickly run experiments on the content. The encoder and decoder can also be reset or reverted to a particular snapshot.
Control of the Pipeline
In addition to allowing you to inspect each stage of the Cairo pipeline, Codec Tool also enables you to modify and re-try parts of the codec. For example, you can change block types, set new quantization matrices, change the transform, update the quality, or toggle entire stages of the pipeline.