Optimizing the B lack Bo x of HTML <video>

1. Optimizing theBlack Bo xof HTML <video> • Will Law | Chief Media Architect | Akamai

2. The video tag seems like the simple solution to ouronline video problems <video controls poster=”myimage.jpg” style="width:480px; height:270px"> <source src=”myfile.mp4" type='video/mp4'/> <source src=”myfile.webm" type='video/webm' /> </video>

3. Let’s take a typical site http://velocityconf.com/

4. and dress it up by adding some video http://mediapm.edgesuite.net/will/presentations/velocitynyc13/preloadnone/oreilly-preload-none.html

5. Problem #1: Embedding a video object on page load nearly always causes additional download penalty. • Add a poster image to the page, in the same size and location as the video • When the user clicks it, hide the poster and dynamically add a video element in its place, with autoplay enabled • If we perform this in the same function that handles the image click then we can autostart the video under iOS and Android. • http://mediapm.edgesuite.net/will/presentations/velocitynyc13/optimize/ondemand2.html Solution: defer instantiation of the <video> object until we need itHow:

6. Problem #2: progressive videos continue downloading when you pause.Solution: remove the video object source on pause.How: • Instantiate the <video> element when the user wants the video • When they click pause, store the playhead position, set the src to “”, remove the element. • Add the element back to the page with the correct src • When they hit play, issue a seek() to the stored position. • http://mediapm.edgesuite.net/will/presentations/velocitynyc13/optimize/ondemand3.html

7. The myth of SBR • Any single bitrate you choose for a video is by definition the wrong video for 99% of users. These users will rebuffer These have lower quality than they could sustain

8. Problem #3: a single bitrate is not optimal for all users.Solution: let’s select an appropriate bitrate before we start delivering the video.How: • When the users loads the page, estimate their throughput by timing the download of a small binary object • Select the appropriate source streams based on this bitrate estimate bitrate and then dynamically add that source to the video object. • http://mediapm.edgesuite.net/will/presentations/velocitynyc13/optimize/measure1.html

9. Problem #4: this timed download is wasteful if the user never watches the video.Solution: defer the bandwidth estimate until the user actually selects the video to watch..How: • When the user selects the video, estimate their throughput by timing the download of a small binary object • Immediately select the appropriate source streams based on this bitrate estimate bitrate and then dynamically add that source to the video object and start playback • http://mediapm.edgesuite.net/will/presentations/velocitynyc13/optimize/measure3.html

10. Bandwidth is never constant • Even preselecting bitrates will lead to rebuffering problems as bandwidth fluctuates over the duration of the vide, especially for long form content • Ideally we would like to switch bitrates constantly to always give the user the highest quality they can sustain at any point in time. Buffering

11. Bandwidth is never constant • Even preselecting bitrates will lead to rebuffering problems as bandwidth fluctuates over the duration of the vide, especially for long form content • Ideally we would like to switch bitrates constantly to always give the user the highest quality they can sustain at any point in time. Adaptive delivery 1500kbps 1000kbps Buffering 500kbps

12. Adaptive media delivery is the notion of constantly changing the bitrate (quality) of the video being delivered to suit the throughput at that point in time. • It’s typically controlled by the client == smart(er) clients • The stream is delivered in segments in enable switching

13. How does segmentation work? into multiple short blocks. Each block holds the same section of video, encoded at a different size and bitrate. 1 3 2 Incoming video 500 kbps 1000 kbps 2000 kbps is split by an encoder

14. How does adaptive delivery work? The segmented video is stored on a server, along with a text file which describes the names of each segment. This text file is called a manifest. A player downloads the manifest and then begins requesting individual segments of video. It makes its choice based on bandwidth conditions, grabbing the best quality it can at the time. SERVER CLOUD PLAYER

15. Which adaptive streaming formats exist today? • HTTP Live Streaming (HDS) • Controlled by Apple • Known as HLS, it is supported well by iOS, Safari and half-heartedly by Android. • HTTP Dynamic Streaming (HDS) • Controlled by Adobe • Known as HDS, it is played back only by Flash clients with custom apps. • Smooth Streaming (SMOOTH) • Controlled by Microsoft • Played back via Silverlight clients, Xbox, • MPEG-DASH (DASH) • An ISO standard • Playback via MSE in browsers, native apps on iOS, Android and WInOS.

16. Simple HLS example <video id="player" controls poster="sintel.jpg" preload="none" style="width:480px;height:204px"> <source src="http://multiplatform-f.akamaihd.net/i/multi/april11/sintel/sintel-hd_,512x288_450_b,640x360_700_b,768x432_1000_b,.mp4.csmil/master.m3u8" type='application/x-mpegURL' /> <source src="http://akamaicorp-pmd.edgesuite.net/flashvod/video/mp4/sintel.mp4" type='video/mp4'/> <source src="http://akamaicorp-pmd.edgesuite.net/flashvod/video/mp4/sintel.webm" type='video/webm' /> </video>

17. Media Source Extension – the future of browser video • A W3C standard - https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html • Supported today by Chrome, IE11 and Opera • MSE allows JavaScript to • fetch and parse a manifest • retrieve video, audio and text segments using XHR • feed them to the <video> tag to be decoded and rendered. MSE gives developers incredible control over playback, buffering and adaption logic of in-page video.

18. dash.js • An open-source project on Github: https://github.com/Dash-Industry-Forum/dash.js • Provides a library for playback of DASH adaptive content using MSE. • Let’s put it all together and make an implementation which • Uses MBR DASH under Chrome/IE11 • Uses MBR HLS under iOS • Uses progressive media with bitrate selection outside of those environments. • Let’s do all this with a 9KB initial page load. • http://mediapm.edgesuite.net/will/presentations/velocitynyc13/adaptive/all.html

19. Take these thoughts home … • Due to its size, video can have a disproportionate effect on page performance and load times. • Try to defer instantiation of video components until the user actually wants to watch • If using progressive video delivery, use some sort of speedtestto pick the best quality stream your user can enjoy at that moment. • Adaptive delivery gives a better user experience, especially for long form content. • MediaSource extensions in HTML5 are the precursors of the browser becoming the dominant video player application environment among devices. Become familiar with them and enjoy the ride. • All code samples available at http://tinyurl.com/willatvelocity2013 Questions?