Randomized Decision Forests

Friday, October 23rd, 2015

Over the last decade we've witnessed a transformation in human interfaces brought on largely by advances in machine learning. Automated phone assistants, voice and text translators, self driving cars, and much more, are all reliant on various forms of machine learning.

The randomized decision forest is a machine learning algorithm that's useful for these kinds of tasks. This algorithm gained significant popularity over the last several years and serves as the heart of the tracking algorithm within the Microsoft's Kinect product.

This article presents a casual introduction to randomized decision forests, follows with a simple example to highlight the process, and lastly discusses important development considerations. This will not be a rigorous discussion and is intended for those with a general interest in the subject without any prior experience.

Continue reading...

Virtual Reality Video Compression

Sunday, October 11th, 2015

Several of the most popular virtual and augmented reality experiences involve video. These videos come in a variety of formats including combinations of stereoscopic 3D, 360° panoramas and spherical views. Unfortunately, these formats place significant strain on our processors, memory, and network bandwidth due to their increased requirements for resolution, framerate and latency.

To cope with this trifecta of video compression challenges, we need a video codec that capitalizes on the potential for increased levels of self similarity and inter-frame correlation within these kinds of videos.

The following is a list of basic mixed-reality centric features that could be integrated into a modern video codec to help achieve higher compression efficiency for VR content, while lowering processor and bandwidth costs.

Continue reading...

Video Quality Graceful Degradation

Sunday, July 5th, 2015

Rate control, in the context of video streaming, is the continual process of measuring a user's network connection quality and adjusting the video quality parameters to ensure a smooth viewing experience. It's difficult to get right, as it must account for several dynamic factors including the encoder and decoder processing performance and unforeseeable fluctuations in network conditions.

Furthermore, due to the pipelined nature of streaming and the bullwhip effect of congestion, recovery systems typically need to over-compensate for fluctuations by abruptly stepping quality down to a level that the network can sustain. Some of these abrupt changes are more noticable than others, due to the nonlinear relationship between bitrate and perceived quality.

Continue reading...

Experiments with the Cairo Video Codec

Saturday, June 27th, 2015

Cairo is a simple video codec that I developed in early 2011 during the later stages of everyAir. The purpose of this codec was to experiment with compression features and learn about their individual impact on efficiency while developing a simple framework for future work.

Video codecs contain a lot of features such as subpixel motion estimation, differential coding, quantization, deblocking, rate control, entropy coding, and many more. Each feature must be adaptively configured to perform well despite abrupt changes in the source video. Balancing these features with overall frame quality and size can be a daunting task, and codecs that do it well will outperform those that don't.

It's been over four years since Cairo was completed and I thought it might be interesting to dust it off and visualize the benefits of some of its features. Specifically, I was interested in comparing the effects of its generic motion compensation, sub-pixel motion compensation, differential coding, and quality controlled quantization.


Continue reading...

The Basics of Image Denoise Filtering

Saturday, April 18th, 2015

Denoising is a very useful process that's used in video compression, computer vision, video capture pipelines, and more. Anywhere that noise may find its way into an image, denoising filters are there to help clean things up. There are many different kinds of noise, and fortunately there are equally many ways of correcting it.

Modern video codecs include denoising filters to mitigate some of the artifacts produced by the encoding process. The Cairo codec is no exception, and uses a simple denoise filter during image decoding to smooth out noise that may find its way into a frame.

As mentioned, there are many different kinds of noise, but I'd like to focus this article on one particular type: salt and pepper noise. This kind of noise presents as speckled light and dark pixels that can give a frame a grainy look and degrade the clarity of the image. Sometimes these artifacts are desirable, as they can mimic film grain or other visual qualities, but when they are introduced by the codec, it's best to rectify them.


Continue reading...

The Making of Balls Away

Friday, March 20th, 2015

Balls Away is a fast paced action game for iOS that pits you against a horde of evil balls bent on the destruction of a small rabbit colony. Leveraging the Vision 3 game engine, Balls Away features a rich 3D world with randomly generated levels and intense gameplay.

For more information about this game, visit its project page. For more information about the game engine that powers Balls Away, check out the Vision 3 project page.

Continue reading...



previous page    |    1    2    3    4    5    |    next page