Sunday, July 5th, 2015
Rate control, in the context of video streaming, is the continual process of measuring a user's network connection quality and adjusting the video quality parameters
to ensure a smooth viewing experience. It's difficult to get right, as it must account for several dynamic factors including the encoder and decoder processing
performance and unforeseeable fluctuations in network conditions.
Furthermore, due to the pipelined nature of streaming and the bullwhip effect of congestion, recovery systems typically need to over-compensate
for fluctuations by abruptly stepping quality down to a level that the network can sustain. Some of these abrupt changes are more noticable than
others, due to the nonlinear relationship between bitrate and perceived quality.
This is a vital component of video streaming services like Amazon Instant Video, Netflix and Hulu because they enable viewers to use their services uninterrupted
despite wildly changing network conditions that may be impacted by all kinds of factors (e.g. the number of users on the network, what those users
are doing, capacity outages, etc).
Standard Behavior
Surprisingly, there does not seem to be a general consensus on the best way to handle the graceful degredation of a video stream, as each of the
aforementioned streaming services offer significantly different user experiences in this area. As an avid user of these services I'm often
painfully aware of their shortcomings, so I've put together a short list of what I would consider the minimum set of graceful degredation features
and behaviors that all large scale video streaming services should offer.
Buffering Percentage |
||
This is extremely important and it's glaringly noticable when absent. Any time the video must be paused in order to buffer
the user must be given a percentage or progress indicator so that they are assured of active progress plus an
estimate of when video playback will resume.
|
||
Multiple Quality Levels |
||
Most services use a precoded matrix of videos at various quality levels for different network levels and resolutions. When
a change in quality is needed, the service is able to simply switch to a different video feed. Alternatively, for live
streaming solutions the video encode parameters may be adjusted to dynamically adjust the video quality.
|
||
Audio Only |
||
When network quality degrades so severely that even the lowest quality video setting is unacceptable,
the stream can switch to a mode commonly known as storyboard. In this mode, the user is visually
alerted that their stream is suffering and the video is presented at a very low framerate (0.25 to 1 frame per second).
|
||
Degrade Quickly, Recover Slowly |
||
Once the decision has been made to lower the video quality level, it's important to drop to a sufficient level that the network can realistically sustain. Prompting the user multiple times to buffer, or repeated drops followed by rapid improvements to video quality should be avoided. |
||
Remain Responsive |
||
The interface should remain responsive even when network quality has significantly suffered. Regardless of whether the system is buffering, the user should always be able to pause, seek, and halt the stream. |
Cherry On Top
In addition to the features above, here are some nice to haves that can significantly improve the user experience. These features were offered in the first release of everyAir and received positive feedback.
Configurable Quality |
||
Allow power users to set a maximum quality level. The 8k display on my 4.7" iPhone 8S
doesn't need to be powered by 8k video, particularly over LTE. Given the small display
size I would prefer a simple 720p stream that saves my bandwidth.
|
||
Drop to Black and White |
||
A black and white stream can save up to 33% of bandwidth versus full color, and many users won't even notice the sudden change.
If this doesn't seem like a significant improvement, consider that converting low quality streams with obvious blocking artifacts
to black and white can salvage a third of the visual quality without an increase in the cost.
|
Wrap Up
Did I miss a critical feature? Feel free to drop me a line and I'll add it to the list :-)