home.aspx
home.aspx
 
. https://medium.com/netflix-techblog/streaming-video-experimentation-at-netflix-visualizing-practical-and-statistical-significance-7117420f4e9a
blog article
STREAMING VIDEO EXPERIMENTATION AT NETFLIX:  VISUALIZING PRACTICAL AND STATISTICAL SIGNIFICANCE
Streaming video experimentation at Netflix seeks to optimize the Quality of Experience (QoE for short) of the videos we stream to our 130 million members around the world. To measure QoE, we look at a wide variety of metrics for each playback session, including play delay; the rates of rebuffers (playback interruptions when the video buffer empties), playback errors, and user-initiated aborts; the average bitrate throughout playback; and Video Multimethod Assessment Fusion, a measure of perceptual video quality developed here at Netflix.Many of our experiments are “systems tests”: short-running (hours to a week) A/B experiments that seek to improve one QoE metric without harming others. For example, we may test the production configuration of the adaptive streaming algorithm, which selects video quality based on device capabilities, resolution limits based on the Netflix plan tier, and time-varying network conditions, against a new parameter configuration that aims to reduce play delay without degrading other metrics. Although each test that results in the rollout of a new production experience may only incrementally improve one or two QoE metrics, and only for some members, over time the cumulative impact is a steady improvement in our ability to efficiently deliver high quality streaming video at scale to all of our diverse members. READ MORE