One of the key innovations in everyAir was a simple video codec designed for low latency streaming. My goal was to ensure that, aside from network induced latency, the video encode and decode steps consumed no more than 8 milliseconds of processing in total (on 2010 PC/mobile hardware). This goal aimed to ensure that the decoding device never became more than a single video frame behind.
I've posted a short paper that details the design of this codec.
In our testing with an iPhone 3GS, we roughly saw costs of 4 ms for full frame decode, and 5 ms for the server side (Intel Core 2 Duos @ 2.2 GHz) CPU encode. With identical content and hardware we observed h.264 decoding costs of 16 ms, and server encoding costs of 18 ms.
In comparison to h.264, bandwidth consumption of p.264 was generally about 1.4x worse, but end-to-end video processing latency was generally about 4x better. This exchange of higher bandwidth for significantly lower latency enabled p.264 to deliver frames much faster and smoother than h.264, but with noticable (though acceptable) degredation in video quality.
Check out the video below to see the codec in action. In this video, an iPhone is overlaid on top of a monitor. Video content is presented on the monitor simultaneously while it is being wirelessly streamed to the iPhone. Although there is visible latency caused by the network, the two videos still appear to be reasonably in sync.
