https://wiki.whatwg.org/api.php?action=feedcontributions&user=Silviapfeiffer&feedformat=atomWHATWG Wiki - User contributions [en]2024-03-29T16:02:00ZUser contributionsMediaWiki 1.39.3https://wiki.whatwg.org/index.php?title=User:Silviapfeiffer&diff=9031User:Silviapfeiffer2013-02-21T08:42:32Z<p>Silviapfeiffer: Created page with "{{CC0 user}}"</p>
<hr />
<div>{{CC0 user}}</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=8516Use cases for timed tracks rendered over video by the UA2012-10-01T11:14:16Z<p>Silviapfeiffer: /* Video Examples */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
This example shows a case where animated positioning and angular change are highly recommended to achieve a desired style: http://www.youtube.com/watch?v=Qz2F-H7IHFM<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
Karaoke with color coded voice, where parts sung together have a unique color ([http://www.youtube.com/watch?v=5tOxfjHLK-A source]):<br />
<br />
<img src="http://bayimg.com/image/iamdlaaci.jpg" title="Jacky sings"><br />
<img src="http://bayimg.com/image/iamdpaaci.jpg" title="Landy sings"><br />
<img src="http://bayimg.com/image/jamdbaaci.jpg" title="Jacky+Landy sing"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
<br />
== Video Examples ==<br />
<br />
See http://www.dcmp.org/captioningkey/types_methods_styles.html and the pages following this one (click on the next topic at the bottom)<br />
<br />
This is now at http://www.dcmp.org/captioningkey/text.html</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=8123Video Metrics2012-05-03T23:49:22Z<p>Silviapfeiffer: added moz bug link</p>
<hr />
<div>'''Related HTML WG bugs:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399 , http://www.w3.org/Bugs/Public/show_bug.cgi?id=14310<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have [https://bugzilla.mozilla.org/show_bug.cgi?id=580531 implemented] the following [https://developer.mozilla.org/en/DOM/HTMLVideoElement statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozFrameDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
Mozilla are also [https://bugzilla.mozilla.org/show_bug.cgi?id=686370 working on some of the statistics listed here].<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''Silverlight''' [http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ advanced logging] relevant stats:<br />
<br />
* c-buffercount: Number of times rebuffering is required. This is how many times we underflow basically. This is calculated at the frame level.<br />
* c-bytes: Number of bytes received by the client from the server. The value does not include any overhead that is added by the network stack. However, HTTP may introduce some overhead. Therefore, the same content streamed by using different protocols may result in different values. If c-bytes and sc-bytes(server-side) are not identical, packet loss occurred.<br />
* c-starttime: The point where the client began watching the stream (in seconds, no fraction). For true live streaming, we need to calculate time offset using wallclock time.<br />
* x-duration: Duration (in seconds) of the data rendered by the client from c-starttime.<br />
* startupTimeMs: From play to render first frame (in milliseconds).<br />
* bandwidthMax: Maximum perceived bandwidth<br />
* bandwidthMin: Minimum perceived bandwidth<br />
* bandwidthAvg: Average perceived bandwidth<br />
* droppedFramesPerSecond: Dropped frames per second (provided by Silverlight)<br />
* renderedFramesPerSecond: Rendered frames per second (provided by Silverlight)<br />
* audioResponseTimeAvg: Average response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMax: Maximum response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMin: Minimum response time to get audio chunks. This is time from request to last byte.<br />
* videoResponseTimeAvg: Average response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMax: Maximum response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMin: Minimum response time to get video chunks. This is time from request to last byte.<br />
* audioDownloadErrors: Total number of missing audio chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs.<br />
* videoDownloadErrors: Total number of missing video chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs<br />
* audioPlaybackBitrates: An ordered list of the audio bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* videoPlaybackBitrates: An ordered list of the video bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* audioBandwidthAvg: Average audio bit rate for the downloaded chunks<br />
* videoBandwidthAvg: Average video bit rate for the downloaded chunks<br />
* audioBufferSizeAvg: Average audio buffer size (in seconds) during playback<br />
* audioBufferSizeMax: Maximum audio buffer size (in seconds) during playback<br />
* videoBufferSizeAvg: Average video buffer size (in seconds) during playback<br />
* videoBufferSizeMax: Maximum video buffer size (in seconds) during playback<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.<br />
<br />
== Proposal ==<br />
<br />
The attributes specified here are intended to contain only those that are not calculable by the application from any other source (or be derivable from themselves). E.g. there is no need for a ‘frames per second’ attribute.<br />
<br />
To be more specific, there are no time based derivatives as any sampling window will never be applicable for all applications - an application developer can use the attribute value to calculate the incoming rate by creating a timer and calculating the difference in values in successive timer calls.<br />
<br />
''Open issue: Do these values reset when the source media has changed?''<br />
<br />
===bytesReceived===<br />
Applicable to: <video>, <audio><br />
<br />
The raw bytes received from the network for decode. Together with the downloadTime, this can be used to calculate the effective bandwidth. Includes container data.<br />
<br />
'''Use case:''' This measures the network performance. This would be used to report the effective video bandwidth received by the browser to the server (i.e. the content publisher) to better understand what bandwidth is typically available to users and encode the videos on the server in the right set of bitrates (e.g. for what bandwidths are typically received in mobile / in desktop environments in different countries, so what should the edge servers be seeded with).<br />
<br />
===downloadTime===<br />
Applicable to: <video>, <audio><br />
<br />
The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).<br />
<br />
'''Use case:''' This allows to determine network performance together with "bytesReceived" by providing what time has actually been spent for downloading the resource (parts). This is better than just starting a timer when putting the resource URL into the video element, since times where the browser does not try to download more parts of the resource are not part of this downloadTime.<br />
<br />
<br />
===networkWaitTime===<br />
Applicable to: <video>, <audio><br />
<br />
The total duration of time when a playback is stalled, waiting to receive more data from network.<br />
<br />
'''Use case:''' This allows to determine how much of the downloadTime is actually waiting time and thus allows to determine situations of network congestion and delays and separate that from the bandwidth that is available to the user.<br />
<br />
<br />
===videoBytesDecoded===<br />
Applicable to: <video><br />
<br />
The number of bytes of video data that have been decoded. This can be used to calculate the effective video bitrate. Does not include container data.<br />
<br />
See [[#bytesdecodedsuggestion|suggestion]] below for a proposed alternative form.<br />
<br />
'''Use case:''' This measures the decoding pipeline performance. This would be used to report whether the decoding pipeline keeps up with the playback position, given that the network feeds sufficient bytes into this pipeline. videoBytesDecoded would measure the performance of the video decoding pipeline.<br />
<br />
<br />
===audioBytesDecoded===<br />
Applicable to: <audio>, <video><br />
<br />
The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.<br />
<br />
'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded. audioBytesDecoded would measure the performance of the audio decoding pipeline.<br />
<br />
:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').<br />
<br />
'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded and audioBytesDecoded, but would be applicable per track (and is therefore probably more generally applicable). It can e.g. help identify slow browsers, or plugins that slow down the decoding, or the optimal number of tracks to put into a file before the decoding engine gets overloaded.<br />
<br />
<br />
===decodedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been decoded and made available for presentation.<br />
<br />
'''Use case:''' This measure the decoding pipeline performance on a frame level rather than a byte level to allow making judgements about the rendering engine when comparing to droppedFrames.<br />
<br />
<br />
===droppedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.<br />
<br />
'''Use case:''' This measures the number of frames that are already too late to be handed to the rendering pipeline after decoding and thus measures the speed of the decoding pipeline.<br />
<br />
<br />
===presentedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.<br />
<br />
'''Use case:''' This measures the display quality that the user perceives and can determine the performance of the rendering engine given the performance of the network and decoding pipeline. If, for example, the system receives sufficient data from the network (no droppedFrames), but the rate of presented frames per second is below 15, we can assume that the user gets a pretty crappy presentation because the rendering engine was too slow, the machine is likely overloaded or not capable of rendering this video at this quality and we should probably move to a lower bitrate (resolution or framerate) resource for the video.<br />
<br />
<br />
===playbackJitter===<br />
Applicable to: <video><br />
<br />
It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:<br />
<br />
; Ei : Desired duration of frame i spent on the screen (to nearest microsecond?)<br />
; Ai : Actual duration frame i spent on the screen (if the frame is never presented to the user, then Ai == 0).<br />
<br />
then:<br />
<br />
:playbackJitter = sum(abs(Ei - Ai))<br />
<br />
The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered (which would normally be presentedFrames + droppedFrames).<br />
<br />
[[Category:Proposals]]</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=8122Video Metrics2012-05-03T23:48:12Z<p>Silviapfeiffer: updating Firefox stats</p>
<hr />
<div>'''Related HTML WG bugs:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399 , http://www.w3.org/Bugs/Public/show_bug.cgi?id=14310<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have [https://bugzilla.mozilla.org/show_bug.cgi?id=580531 implemented] the following [https://developer.mozilla.org/en/DOM/HTMLVideoElement statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozFrameDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''Silverlight''' [http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ advanced logging] relevant stats:<br />
<br />
* c-buffercount: Number of times rebuffering is required. This is how many times we underflow basically. This is calculated at the frame level.<br />
* c-bytes: Number of bytes received by the client from the server. The value does not include any overhead that is added by the network stack. However, HTTP may introduce some overhead. Therefore, the same content streamed by using different protocols may result in different values. If c-bytes and sc-bytes(server-side) are not identical, packet loss occurred.<br />
* c-starttime: The point where the client began watching the stream (in seconds, no fraction). For true live streaming, we need to calculate time offset using wallclock time.<br />
* x-duration: Duration (in seconds) of the data rendered by the client from c-starttime.<br />
* startupTimeMs: From play to render first frame (in milliseconds).<br />
* bandwidthMax: Maximum perceived bandwidth<br />
* bandwidthMin: Minimum perceived bandwidth<br />
* bandwidthAvg: Average perceived bandwidth<br />
* droppedFramesPerSecond: Dropped frames per second (provided by Silverlight)<br />
* renderedFramesPerSecond: Rendered frames per second (provided by Silverlight)<br />
* audioResponseTimeAvg: Average response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMax: Maximum response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMin: Minimum response time to get audio chunks. This is time from request to last byte.<br />
* videoResponseTimeAvg: Average response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMax: Maximum response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMin: Minimum response time to get video chunks. This is time from request to last byte.<br />
* audioDownloadErrors: Total number of missing audio chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs.<br />
* videoDownloadErrors: Total number of missing video chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs<br />
* audioPlaybackBitrates: An ordered list of the audio bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* videoPlaybackBitrates: An ordered list of the video bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* audioBandwidthAvg: Average audio bit rate for the downloaded chunks<br />
* videoBandwidthAvg: Average video bit rate for the downloaded chunks<br />
* audioBufferSizeAvg: Average audio buffer size (in seconds) during playback<br />
* audioBufferSizeMax: Maximum audio buffer size (in seconds) during playback<br />
* videoBufferSizeAvg: Average video buffer size (in seconds) during playback<br />
* videoBufferSizeMax: Maximum video buffer size (in seconds) during playback<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.<br />
<br />
== Proposal ==<br />
<br />
The attributes specified here are intended to contain only those that are not calculable by the application from any other source (or be derivable from themselves). E.g. there is no need for a ‘frames per second’ attribute.<br />
<br />
To be more specific, there are no time based derivatives as any sampling window will never be applicable for all applications - an application developer can use the attribute value to calculate the incoming rate by creating a timer and calculating the difference in values in successive timer calls.<br />
<br />
''Open issue: Do these values reset when the source media has changed?''<br />
<br />
===bytesReceived===<br />
Applicable to: <video>, <audio><br />
<br />
The raw bytes received from the network for decode. Together with the downloadTime, this can be used to calculate the effective bandwidth. Includes container data.<br />
<br />
'''Use case:''' This measures the network performance. This would be used to report the effective video bandwidth received by the browser to the server (i.e. the content publisher) to better understand what bandwidth is typically available to users and encode the videos on the server in the right set of bitrates (e.g. for what bandwidths are typically received in mobile / in desktop environments in different countries, so what should the edge servers be seeded with).<br />
<br />
===downloadTime===<br />
Applicable to: <video>, <audio><br />
<br />
The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).<br />
<br />
'''Use case:''' This allows to determine network performance together with "bytesReceived" by providing what time has actually been spent for downloading the resource (parts). This is better than just starting a timer when putting the resource URL into the video element, since times where the browser does not try to download more parts of the resource are not part of this downloadTime.<br />
<br />
<br />
===networkWaitTime===<br />
Applicable to: <video>, <audio><br />
<br />
The total duration of time when a playback is stalled, waiting to receive more data from network.<br />
<br />
'''Use case:''' This allows to determine how much of the downloadTime is actually waiting time and thus allows to determine situations of network congestion and delays and separate that from the bandwidth that is available to the user.<br />
<br />
<br />
===videoBytesDecoded===<br />
Applicable to: <video><br />
<br />
The number of bytes of video data that have been decoded. This can be used to calculate the effective video bitrate. Does not include container data.<br />
<br />
See [[#bytesdecodedsuggestion|suggestion]] below for a proposed alternative form.<br />
<br />
'''Use case:''' This measures the decoding pipeline performance. This would be used to report whether the decoding pipeline keeps up with the playback position, given that the network feeds sufficient bytes into this pipeline. videoBytesDecoded would measure the performance of the video decoding pipeline.<br />
<br />
<br />
===audioBytesDecoded===<br />
Applicable to: <audio>, <video><br />
<br />
The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.<br />
<br />
'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded. audioBytesDecoded would measure the performance of the audio decoding pipeline.<br />
<br />
:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').<br />
<br />
'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded and audioBytesDecoded, but would be applicable per track (and is therefore probably more generally applicable). It can e.g. help identify slow browsers, or plugins that slow down the decoding, or the optimal number of tracks to put into a file before the decoding engine gets overloaded.<br />
<br />
<br />
===decodedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been decoded and made available for presentation.<br />
<br />
'''Use case:''' This measure the decoding pipeline performance on a frame level rather than a byte level to allow making judgements about the rendering engine when comparing to droppedFrames.<br />
<br />
<br />
===droppedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.<br />
<br />
'''Use case:''' This measures the number of frames that are already too late to be handed to the rendering pipeline after decoding and thus measures the speed of the decoding pipeline.<br />
<br />
<br />
===presentedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.<br />
<br />
'''Use case:''' This measures the display quality that the user perceives and can determine the performance of the rendering engine given the performance of the network and decoding pipeline. If, for example, the system receives sufficient data from the network (no droppedFrames), but the rate of presented frames per second is below 15, we can assume that the user gets a pretty crappy presentation because the rendering engine was too slow, the machine is likely overloaded or not capable of rendering this video at this quality and we should probably move to a lower bitrate (resolution or framerate) resource for the video.<br />
<br />
<br />
===playbackJitter===<br />
Applicable to: <video><br />
<br />
It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:<br />
<br />
; Ei : Desired duration of frame i spent on the screen (to nearest microsecond?)<br />
; Ai : Actual duration frame i spent on the screen (if the frame is never presented to the user, then Ai == 0).<br />
<br />
then:<br />
<br />
:playbackJitter = sum(abs(Ei - Ai))<br />
<br />
The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered (which would normally be presentedFrames + droppedFrames).<br />
<br />
[[Category:Proposals]]</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=7275Video Metrics2011-10-11T04:03:21Z<p>Silviapfeiffer: added use cases for metrics</p>
<hr />
<div>'''Related HTML WG bugs:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399 , http://www.w3.org/Bugs/Public/show_bug.cgi?id=14310<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''Silverlight''' [http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ advanced logging] relevant stats:<br />
<br />
* c-buffercount: Number of times rebuffering is required. This is how many times we underflow basically. This is calculated at the frame level.<br />
* c-bytes: Number of bytes received by the client from the server. The value does not include any overhead that is added by the network stack. However, HTTP may introduce some overhead. Therefore, the same content streamed by using different protocols may result in different values. If c-bytes and sc-bytes(server-side) are not identical, packet loss occurred.<br />
* c-starttime: The point where the client began watching the stream (in seconds, no fraction). For true live streaming, we need to calculate time offset using wallclock time.<br />
* x-duration: Duration (in seconds) of the data rendered by the client from c-starttime.<br />
* startupTimeMs: From play to render first frame (in milliseconds).<br />
* bandwidthMax: Maximum perceived bandwidth<br />
* bandwidthMin: Minimum perceived bandwidth<br />
* bandwidthAvg: Average perceived bandwidth<br />
* droppedFramesPerSecond: Dropped frames per second (provided by Silverlight)<br />
* renderedFramesPerSecond: Rendered frames per second (provided by Silverlight)<br />
* audioResponseTimeAvg: Average response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMax: Maximum response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMin: Minimum response time to get audio chunks. This is time from request to last byte.<br />
* videoResponseTimeAvg: Average response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMax: Maximum response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMin: Minimum response time to get video chunks. This is time from request to last byte.<br />
* audioDownloadErrors: Total number of missing audio chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs.<br />
* videoDownloadErrors: Total number of missing video chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs<br />
* audioPlaybackBitrates: An ordered list of the audio bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* videoPlaybackBitrates: An ordered list of the video bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* audioBandwidthAvg: Average audio bit rate for the downloaded chunks<br />
* videoBandwidthAvg: Average video bit rate for the downloaded chunks<br />
* audioBufferSizeAvg: Average audio buffer size (in seconds) during playback<br />
* audioBufferSizeMax: Maximum audio buffer size (in seconds) during playback<br />
* videoBufferSizeAvg: Average video buffer size (in seconds) during playback<br />
* videoBufferSizeMax: Maximum video buffer size (in seconds) during playback<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.<br />
<br />
<br />
== Proposal ==<br />
<br />
The attributes specified here are intended to contain only those that are not calculable by the application from any other source (or be derivable from themselves). E.g. there is no need for a ‘frames per second’ attribute.<br />
<br />
To be more specific, there are no time based derivatives as any sampling window will never be applicable for all applications - an application developer can use the attribute value to calculate the incoming rate by creating a timer and calculating the difference in values in successive timer calls.<br />
<br />
''Open issue: Do these values reset when the source media has changed?''<br />
<br />
===bytesReceived===<br />
Applicable to: <video>, <audio><br />
<br />
The raw bytes received from the network for decode. Together with the downloadTime, this can be used to calculate the effective bandwidth. Includes container data.<br />
<br />
'''Use case:''' This measures the network performance. This would be used to report the effective video bandwidth received by the browser to the server (i.e. the content publisher) to better understand what bandwidth is typically available to users and encode the videos on the server in the right set of bitrates (e.g. for what bandwidths are typically received in mobile / in desktop environments in different countries, so what should the edge servers be seeded with).<br />
<br />
===downloadTime===<br />
Applicable to: <video>, <audio><br />
<br />
The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).<br />
<br />
'''Use case:''' This allows to determine network performance together with "bytesReceived" by providing what time has actually been spent for downloading the resource (parts). This is better than just starting a timer when putting the resource URL into the video element, since times where the browser does not try to download more parts of the resource are not part of this downloadTime.<br />
<br />
<br />
===networkWaitTime===<br />
Applicable to: <video>, <audio><br />
<br />
The total duration of time when a playback is stalled, waiting to receive more data from network.<br />
<br />
'''Use case:''' This allows to determine how much of the downloadTime is actually waiting time and thus allows to determine situations of network congestion and delays and separate that from the bandwidth that is available to the user.<br />
<br />
<br />
===videoBytesDecoded===<br />
Applicable to: <video><br />
<br />
The number of bytes of video data that have been decoded. This can be used to calculate the effective video bitrate. Does not include container data.<br />
<br />
See [[#bytesdecodedsuggestion|suggestion]] below for a proposed alternative form.<br />
<br />
'''Use case:''' This measures the decoding pipeline performance. This would be used to report whether the decoding pipeline keeps up with the playback position, given that the network feeds sufficient bytes into this pipeline. videoBytesDecoded would measure the performance of the video decoding pipeline.<br />
<br />
<br />
===audioBytesDecoded===<br />
Applicable to: <audio>, <video><br />
<br />
The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.<br />
<br />
'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded. audioBytesDecoded would measure the performance of the audio decoding pipeline.<br />
<br />
:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').<br />
<br />
'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded and audioBytesDecoded, but would be applicable per track (and is therefore probably more generally applicable). It can e.g. help identify slow browsers, or plugins that slow down the decoding, or the optimal number of tracks to put into a file before the decoding engine gets overloaded.<br />
<br />
<br />
===decodedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been decoded and made available for presentation.<br />
<br />
'''Use case:''' This measure the decoding pipeline performance on a frame level rather than a byte level to allow making judgements about the rendering engine when comparing to droppedFrames.<br />
<br />
<br />
===droppedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.<br />
<br />
'''Use case:''' This measures the number of frames that are already too late to be handed to the rendering pipeline after decoding and thus measures the speed of the decoding pipeline.<br />
<br />
<br />
===presentedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.<br />
<br />
'''Use case:''' This measures the display quality that the user perceives and can determine the performance of the rendering engine given the performance of the network and decoding pipeline. If, for example, the system receives sufficient data from the network (no droppedFrames), but the rate of presented frames per second is below 15, we can assume that the user gets a pretty crappy presentation because the rendering engine was too slow, the machine is likely overloaded or not capable of rendering this video at this quality and we should probably move to a lower bitrate (resolution or framerate) resource for the video.<br />
<br />
<br />
===playbackJitter===<br />
Applicable to: <video><br />
<br />
It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:<br />
<br />
; Ei : Desired duration of frame i spent on the screen (to nearest microsecond?)<br />
; Ai : Actual duration frame i spent on the screen (if the frame is never presented to the user, then Ai == 0).<br />
<br />
then:<br />
<br />
:playbackJitter = sum(abs(Ei - Ai))<br />
<br />
The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered (which would normally be presentedFrames + droppedFrames).<br />
<br />
[[Category:Proposals]]</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=7220Video Metrics2011-10-05T01:44:12Z<p>Silviapfeiffer: added metworkWaitTime</p>
<hr />
<div>'''Related HTML WG bugs:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399 , http://www.w3.org/Bugs/Public/show_bug.cgi?id=14310<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''Silverlight''' [http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ advanced logging] relevant stats:<br />
<br />
* c-buffercount: Number of times rebuffering is required. This is how many times we underflow basically. This is calculated at the frame level.<br />
* c-bytes: Number of bytes received by the client from the server. The value does not include any overhead that is added by the network stack. However, HTTP may introduce some overhead. Therefore, the same content streamed by using different protocols may result in different values. If c-bytes and sc-bytes(server-side) are not identical, packet loss occurred.<br />
* c-starttime: The point where the client began watching the stream (in seconds, no fraction). For true live streaming, we need to calculate time offset using wallclock time.<br />
* x-duration: Duration (in seconds) of the data rendered by the client from c-starttime.<br />
* startupTimeMs: From play to render first frame (in milliseconds).<br />
* bandwidthMax: Maximum perceived bandwidth<br />
* bandwidthMin: Minimum perceived bandwidth<br />
* bandwidthAvg: Average perceived bandwidth<br />
* droppedFramesPerSecond: Dropped frames per second (provided by Silverlight)<br />
* renderedFramesPerSecond: Rendered frames per second (provided by Silverlight)<br />
* audioResponseTimeAvg: Average response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMax: Maximum response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMin: Minimum response time to get audio chunks. This is time from request to last byte.<br />
* videoResponseTimeAvg: Average response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMax: Maximum response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMin: Minimum response time to get video chunks. This is time from request to last byte.<br />
* audioDownloadErrors: Total number of missing audio chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs.<br />
* videoDownloadErrors: Total number of missing video chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs<br />
* audioPlaybackBitrates: An ordered list of the audio bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* videoPlaybackBitrates: An ordered list of the video bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* audioBandwidthAvg: Average audio bit rate for the downloaded chunks<br />
* videoBandwidthAvg: Average video bit rate for the downloaded chunks<br />
* audioBufferSizeAvg: Average audio buffer size (in seconds) during playback<br />
* audioBufferSizeMax: Maximum audio buffer size (in seconds) during playback<br />
* videoBufferSizeAvg: Average video buffer size (in seconds) during playback<br />
* videoBufferSizeMax: Maximum video buffer size (in seconds) during playback<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.<br />
<br />
<br />
== Proposal ==<br />
<br />
The attributes specified here are intended to contain only those that are not calculable by the application from any other source (or be derivable from themselves). E.g. there is no need for a ‘frames per second’ attribute.<br />
<br />
To be more specific, there are no time based derivatives as any sampling window will never be applicable for all applications - an application developer can use the attribute value to calculate the incoming rate by creating a timer and calculating the difference in values in successive timer calls.<br />
<br />
''Open issue: Do these values reset when the source media has changed?''<br />
<br />
===bytesReceived===<br />
Applicable to: <video>, <audio><br />
<br />
The raw bytes received from the network for decode. This can be used to calculate the effective bandwidth. Includes container data.<br />
<br />
===videoBytesDecoded===<br />
Applicable to: <video><br />
<br />
The number of bytes of video data that have been decoded. This can be used to calculate the effective video bitrate. Does not include container data.<br />
<br />
See [[#bytesdecodedsuggestion|suggestion]] below for a proposed alternative form.<br />
<br />
===audioBytesDecoded===<br />
Applicable to: <audio>, <video><br />
<br />
The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.<br />
<br />
:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').<br />
<br />
===decodedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been decoded and made available for presentation.<br />
<br />
===droppedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.<br />
<br />
===presentedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.<br />
<br />
===downloadTime===<br />
Applicable to:<video>,<audio><br />
The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).<br />
<br />
===networkWaitTime===<br />
The total duration of time when a playback is stalled, waiting to receive more data from network.<br />
<br />
===playbackJitter===<br />
Applicable to: <video><br />
<br />
It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:<br />
<br />
; Ei : Desired duration of frame i spent on the screen (to nearest microsecond?)<br />
; Ai : Actual duration frame i spent on the screen (if the frame is never presented to the user, then Ai == 0).<br />
<br />
then:<br />
<br />
:playbackJitter = sum(abs(Ei - Ai))<br />
<br />
The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered (which would normally be presentedFrames + droppedFrames).<br />
<br />
[[Category:Proposals]]</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=7219Video Metrics2011-10-05T01:40:01Z<p>Silviapfeiffer: added downloadTime</p>
<hr />
<div>'''Related HTML WG bugs:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399 , http://www.w3.org/Bugs/Public/show_bug.cgi?id=14310<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''Silverlight''' [http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ advanced logging] relevant stats:<br />
<br />
* c-buffercount: Number of times rebuffering is required. This is how many times we underflow basically. This is calculated at the frame level.<br />
* c-bytes: Number of bytes received by the client from the server. The value does not include any overhead that is added by the network stack. However, HTTP may introduce some overhead. Therefore, the same content streamed by using different protocols may result in different values. If c-bytes and sc-bytes(server-side) are not identical, packet loss occurred.<br />
* c-starttime: The point where the client began watching the stream (in seconds, no fraction). For true live streaming, we need to calculate time offset using wallclock time.<br />
* x-duration: Duration (in seconds) of the data rendered by the client from c-starttime.<br />
* startupTimeMs: From play to render first frame (in milliseconds).<br />
* bandwidthMax: Maximum perceived bandwidth<br />
* bandwidthMin: Minimum perceived bandwidth<br />
* bandwidthAvg: Average perceived bandwidth<br />
* droppedFramesPerSecond: Dropped frames per second (provided by Silverlight)<br />
* renderedFramesPerSecond: Rendered frames per second (provided by Silverlight)<br />
* audioResponseTimeAvg: Average response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMax: Maximum response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMin: Minimum response time to get audio chunks. This is time from request to last byte.<br />
* videoResponseTimeAvg: Average response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMax: Maximum response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMin: Minimum response time to get video chunks. This is time from request to last byte.<br />
* audioDownloadErrors: Total number of missing audio chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs.<br />
* videoDownloadErrors: Total number of missing video chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs<br />
* audioPlaybackBitrates: An ordered list of the audio bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* videoPlaybackBitrates: An ordered list of the video bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* audioBandwidthAvg: Average audio bit rate for the downloaded chunks<br />
* videoBandwidthAvg: Average video bit rate for the downloaded chunks<br />
* audioBufferSizeAvg: Average audio buffer size (in seconds) during playback<br />
* audioBufferSizeMax: Maximum audio buffer size (in seconds) during playback<br />
* videoBufferSizeAvg: Average video buffer size (in seconds) during playback<br />
* videoBufferSizeMax: Maximum video buffer size (in seconds) during playback<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.<br />
<br />
<br />
== Proposal ==<br />
<br />
The attributes specified here are intended to contain only those that are not calculable by the application from any other source (or be derivable from themselves). E.g. there is no need for a ‘frames per second’ attribute.<br />
<br />
To be more specific, there are no time based derivatives as any sampling window will never be applicable for all applications - an application developer can use the attribute value to calculate the incoming rate by creating a timer and calculating the difference in values in successive timer calls.<br />
<br />
''Open issue: Do these values reset when the source media has changed?''<br />
<br />
===bytesReceived===<br />
Applicable to: <video>, <audio><br />
<br />
The raw bytes received from the network for decode. This can be used to calculate the effective bandwidth. Includes container data.<br />
<br />
===videoBytesDecoded===<br />
Applicable to: <video><br />
<br />
The number of bytes of video data that have been decoded. This can be used to calculate the effective video bitrate. Does not include container data.<br />
<br />
See [[#bytesdecodedsuggestion|suggestion]] below for a proposed alternative form.<br />
<br />
===audioBytesDecoded===<br />
Applicable to: <audio>, <video><br />
<br />
The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.<br />
<br />
:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').<br />
<br />
===decodedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been decoded and made available for presentation.<br />
<br />
===droppedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.<br />
<br />
===presentedFrames===<br />
Applicable to: <video><br />
<br />
The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.<br />
<br />
===downloadTime===<br />
Applicable to:<video>,<audio><br />
The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).<br />
<br />
===playbackJitter===<br />
Applicable to: <video><br />
<br />
It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:<br />
<br />
; Ei : Desired duration of frame i spent on the screen (to nearest microsecond?)<br />
; Ai : Actual duration frame i spent on the screen (if the frame is never presented to the user, then Ai == 0).<br />
<br />
then:<br />
<br />
:playbackJitter = sum(abs(Ei - Ai))<br />
<br />
The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered (which would normally be presentedFrames + droppedFrames).<br />
<br />
[[Category:Proposals]]</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Adaptive_Streaming&diff=6385Adaptive Streaming2011-04-29T05:07:00Z<p>Silviapfeiffer: added url to encoding params for adaptive streaming</p>
<hr />
<div>Here is a (rough and incomplete) proposal for doing adaptive streaming using open video formats. Key components of the proposal:<br />
<br />
* Videos are served as separate, small chunks.<br />
* Accompanying manifest files provide metadata.<br />
* The user-agent parses manifests and switches between stream levels.<br />
* An API provides QOS metrics and enables custom switching logic.<br />
<br />
== Introduction ==<br />
<br />
Today, most video on the internet is delivered as progressive download (e.g. Youtube). While this works fine in most cases, there are limitations as it comes to more ''advanced'' uses of video:<br />
<br />
* Long-form video (long downloads, waste of bandwidth if user doesn't watch)<br />
* Live/DVR video (hard to do as progressive download, unstable)<br />
* Delivery to mobile devices (lots of buffering due to changing network conditions)<br />
<br />
Adaptive streaming aims to solve these issues by:<br />
<br />
* Offering multiple versions of a video, at different bitrates / quality levels (e.g. from 100kbps to 2 mbps).<br />
* Transporting the video not as one big file, but as separate, distinct chunks (e.g. by cutting up the video in small files, or by using range-requests).<br />
* Allowing user-agents to seamlessly switch between quality levels (e.g. based upon changing device or network conditions), simply by downloading the ''next'' chunk from a different level.<br />
<br />
There's currently three widely used implementations of adaptive HTTP streaming:<br />
<br />
* [http://learn.iis.net/page.aspx/627/smooth-streaming/ Microsoft Smooth Streaming], used by Silverlight.<br />
* [http://www.adobe.com/products/httpdynamicstreaming/ Adobe HTTP Dynamic Streaming], used by Flash.<br />
* [http://devworld.apple.com/resources/http-streaming/ Apple HTTP Live Streaming], used by Quicktime X.<br />
<br />
There's of course still the dedicated streaming protocols (e.g. RTSP over UDP). Adaptive HTTP streaming does not aim to replace those, as there are various cases that are much better served with a ''real'' streaming approach. However, adaptive HTTP streaming could become the approach for the majority of online video delivery, for many of the same reasons that made progressive HTTP so popular:<br />
<br />
* It is easy to understand and implement.<br />
* It builds upon existing HTTP infrastructure.<br />
* It centralizes all intelligence (and control) in the client.<br />
<br />
Note: [http://www.streaminglearningcenter.com/blogs/encoding-for-adaptive-streaming---an-overview.html good article on what matters in your encoding parameters for HTTP adaptive streaming]<br />
<br />
== Chunks ==<br />
<br />
Every chunk should be a valid video file (header, videotrack, audiotrack). Every chunk should also contain at least 1 keyframe (at the start). This implies every single chunk can be played back by itself.<br />
<br />
Beyond validity, the amount of metadata should be kept as small as possible (single-digit kbps overhead).<br />
<br />
Codec parameters that can vary between the different quality levels of an adaptive stream are:<br />
<br />
* The datarate, dimensions (pixel+display) and framerate of the video track.<br />
* The datarate, number of channels and sample frequency of the audio track.<br />
<br />
In order for quality level switches to occur without artifacts, the start positions of all chunks should align between the various quality levels. If this isn't the case, user-agents will display artifacts (ticks, skips, black) when a quality level switch occurs. Syncing should not be a requirement though. This will allow legacy content to be used for dynamic streaming with little effort (e.g. remuxing or using a smart server) and little issues (in practive, most keyframes are aligned between different transcodes of a video).<br />
<br />
In its most low-tech form, chunks can be stored as separate files-on-disc on a webserver. This poses issues around transcoding (no ecosystem yet) and file management (not everybody loves 100s of files). There are at least two solutions:<br />
<br />
# A serverside module accepts chunk requests, pulls the correct GOP(s) from a video and wraps the necessary metadata.<br />
# Clients can accept and process video without headers and use HTTP range-requests to directly get the video data.<br />
<br />
== Manifests ==<br />
<br />
The M3U8 manifest format [http://tools.ietf.org/html/draft-pantos-http-live-streaming-04 that Apple specified] is adopted. Generally, both an overall manifest (linking to the various quality levels) and a quality level manifest (linking to the various stream levels) are used. (Though, especially for live streaming, a single quality level may be used).<br />
<br />
Here's an example of such an overall manifest. It specifies three quality levels, each with its own datarate, codecs and dimensions:<br />
<br />
#EXTM3U <br />
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000,CODECS="vp8,vorbis",RESOLUTION=240x135<br />
http://example.com/low.m3u8 <br />
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2560000,CODECS="vp8,vorbis",RESOLUTION=640x360<br />
http://example.com/mid.m3u8 <br />
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=7680000,CODECS="vp8,vorbis",RESOLUTION=1280x720<br />
http://example.com/hi.m3u8<br />
<br />
Here's an example manifest for one such quality level. It contains a full URL listing of all chunks for this quality level:<br />
<br />
#EXTM3U<br />
#EXT-X-MEDIA-SEQUENCE:0<br />
#EXT-X-TARGETDURATION:10<br />
#EXTINF:10,<br />
http://media.example.com/segment1.webm<br />
#EXTINF:10,<br />
http://media.example.com/segment2.webm<br />
#EXTINF:10,<br />
http://media.example.com/segment3.webm<br />
#EXT-X-ENDLIST<br />
<br />
The video framerate, audio sample frequency and number of audio channels cannot be listed here according to the specs. In formats like WebM, Ogg and MPEG-TS (the container Apple specifies), this can be retrieved during demuxing.<br />
<br />
The #EXT-X-ENDLIST tag defines the end of a video. If this tag is present, the manifest is supposed to be fixed and the client will not re-load it during playback. If this tag is not present (generally only during live streams), the client should periodically re-fetch the manifest to get additional chunks.<br />
<br />
The M3U8 playlist format also provides a mechanism for stream interruptions (#EXT-X-DISCONTINUITY) and for encryption (#EXT-X-KEY). Moreover, regular ID3 tags can be used to enrich the manifest with metadata.<br />
<br />
== User-agents ==<br />
<br />
The root manifest serves as the single, unique reference point for a adaptive stream. Therefore, user agents in theory need solely its URL to playback the stream. Here's an example for loading a root manifest: through the *src* attribute of the <video> tag in an HTML page:<br />
<br />
<video width="480" height="270" src="http://example.com/video.m3u8"><br />
<a href="http://example.com/video_low.webm">Download the video</a><br />
</video><br />
<br />
In this variation, the manifest is loaded through the <source> tag, to provide fallback logic:<br />
<br />
<video width="480" height="270" ><br />
<source src="http://example.com/video-webm.m3u8" type="manifest/webm"><br />
<source src="http://example.com/video-apple.m3u8" type="manifest/m2ts"><br />
<source src="http://example.com/video-plain.webm" type="video/webm"><br />
<a href="http://example.com/video-plain.webm">Download the video</a><br />
</video><br />
<br />
Here's another example for loading the manifest; through the *enclosure* element in an RSS feed:<br />
<br />
<rss version="2.0"><br />
<channel><br />
<title>Example feed</title><br />
<link>http://example.com/</link><br />
<description>Example feed with a single adaptive stream.</description><br />
<item><br />
<title>Example stream</title><br />
<enclosure length="1487" type="manifest/webm"<br />
url="http://example.com/video.m3u8" /><br />
</item><br />
</channel><br />
</rss><br />
<br />
Like the manifest parsing, the switching heuristics are upon the user-agent. They can be somewhat of a *secret sauce*. As a basic example, a user-agent can select a quality level if:<br />
<br />
* The ''bitrate'' of the level is < 90% of the server » client ''downloadRate''.<br />
* The ''videoWidth'' of the level is < 120% of the video element ''width''.<br />
* The delta in ''droppedFrames'' is < 25% of the delta in ''decodedFrames'' for this level.<br />
<br />
Since droppedFrames are only known after a level has started playing, it is generally only a reason for switching down. Based upon the growth rate of droppedFrames, a user-agent might choose to blacklist the quality level for a certain amount of time, or discard it altogether for this playback session.<br />
<br />
The quality level selection occurs at the start of every chunk URL fetch. Given an array of levels, the user-agent starts with the highest quality level first and then walks down the list. If the lowest-quality level does not match the criteria, the user-agent still uses it (else there would be no video).<br />
<br />
A user-agent typically tries to maintain X (3, 10, 20) seconds of video ready for decoding (buffered). If less than X seconds is available, the user-agent runs it quality level selection and requests another chunk.<br />
<br />
There is a tie-in between the length of a chunk, the bufferLenght and the speed with which a user-agent adapts to changing conditions. For example, should the bandwidth drop dramatically, 1 or 2 high-quality chunks will still be played from buffer before the first lower-quality chunk is shown. The other way around is also true: should a user go fullscreen, it will take some time until the stream switches to high quality. Lower bufferLenghts increase responsiveness but also increase the possiblity of buffer underruns.<br />
<br />
== Scripting ==<br />
<br />
Certain user-agents might not offer access to adaptive streaming heuristics. Other user-agents might, or should even do so. The obvious case is a webbrowser supporting the <video> element and a javascript engine:<br />
<br />
=== QOS Metrics ===<br />
<br />
The video element should provides accessors for retrieving quality of service metrics:<br />
<br />
* '''downloadRate''': The current server-client bandwidth (read-only).<br />
* '''videoBitrate''': The current video bitrate (read-only).<br />
* '''droppedFrames''': The total number of frames dropped for this playback session (read-only).<br />
* '''decodedFrames''': The total number of frames decoded for this playback session (read-only).<br />
* '''height''': The current height of the video element (already exists).<br />
* '''videoHeight''': The current height of the videofile (already exists).<br />
* '''width''': The current width of the video element (already exists).<br />
* '''videoWidth''': The current width of the videofile (already exists).<br />
<br />
=== Native adaptive streaming ===<br />
<br />
In case a user-agent has manifest parsing / level switching heuristics built-in, the video element can provides access to the stream levels:<br />
<br />
* '''currentLevel''': The currently playing stream level.<br />
* '''levels''': An array of all stream levels (as parsed from the manifests). Example:<br />
<br />
[{<br />
bitrate: 100000, <br />
codecs: 'vp8,vorbis',<br />
duration: 132,<br />
height: 180,<br />
url: manifest_100.m3u8,<br />
width: 240<br />
},{<br />
bitrate: 500000,<br />
codecs: 'vp8,vorbis',<br />
duration: 132,<br />
height: 360,<br />
url: manifest_500.m3u8,<br />
width: 640<br />
}]<br />
<br />
In addition to this, the video element provides an event to notify scripts of changes in the current stream level:<br />
<br />
* '''levelChange''': the currentLevel attribute has just been updated.<br />
<br />
Last, the video element provides functionality to override the user agent's built-in heuristics:<br />
<br />
* '''setLevel(level)''': This method forces the user to switch to another stream level. Invoking this method disables a user-agent's adaptive streaming heuristics. Use *setLevel(-1)* to enable heuristics again.<br />
* '''bufferLength''': This attribute controls how much videodata (in seconds) a user-agent should strive to keep buffered.<br />
<br />
An important example for '''bufferLength''': a website owner might set this to a very high value to enable viewers on a low bandwidth to wait for buffering and still see a high-quality video.<br />
<br />
<br />
=== API adaptive streaming ===<br />
<br />
In case a user-agent does not have manifest parsing and level switching heuristics built-in, the video element can still accomodate adaptive streaming through a small stream API:<br />
<br />
* '''appendVideo(url,[range])''': fetch the URL and append the video to the currently playing video.<br />
* '''bufferlength''' (read-only): get the current video buffer amount, in seconds.<br />
<br />
The single call allows developers to build adaptive HTTP streaming inside the javascripting layer. Manifest parsing and stream level APIs are not needed. The quality of service metrics are still needed though:<br />
<br />
* When the video plays fine, chunks fro the same quality level are constantly appended.<br />
* When a switch to a different quality level is made, chunks from a different quality level are appended.<br />
* When the user seeks to a different position in the video, it's '''src''' is simply set to the appropriate chunk at that position.<br />
<br />
The '''bufferlength''' getter reports upon the actual amount of data that's in the buffer. Scripts cannot presume any video they append to the videoElement is immediately available: the URL has to be resolved, the data has to be fetched and the data has to be demuxed. <br />
<br />
The '''appendVideo''' call implies that properties such as ''duration'', ''videoHeight'' and ''videoWidth'' may change during a playback session.<br />
<br />
A number of rules have to be set up as to how the concatenation will actually work. For example:<br />
<br />
* In order to allow user-agents to use a single decoding pipeline, the current video and the one that's appended should contain the same container format and A/V codecs.<br />
* Video is appended on a frame-by-frame basis (no bytedata).<br />
* [Audio is appended by slightly extending the data and applying a crossfade?]<br />
<br />
The optional ''range'' parameter instructs the user-agent to only request a certain byterange.<br />
<br />
== Rationale ==<br />
<br />
Finally, some rationale for the choices made in this proposal. Why chunks and a manifest? Why not, for example, range-requests and <source> tags?<br />
<br />
First and foremost, we need a format that works not only in HTML5 browsers, but also in, for example, mobile apps (Android/Blackberry/iOS), desktop players (Miro/Quicktime/VLC) and big screen devices (Roku, Boxee, PS3). Especially for the very small screens (3G network) and large screens (full HD), adaptive streaming is incredibly valuable. Tayloring a solution too much towards the HTML5 syntax and browser environment will hinder broad adoption of an open video standard. Adaptive streaming and HTML5 should work nice together, but adaptive streaming should not be relying on HTML5.<br />
<br />
That said:<br />
<br />
* Providing the low-tech scenario of storing chunks as separate files on the webserver enables adaptive streaming in cases where either the server, the user-agent (apps / players / settops) or the network (firewalls, cellulars) does not support something like range-requests. As an example, implementing adaptive streaming using range-requests in Adobe Flash (e.g. as temporary fallback) would not be possible, since the range-request header is blocked.<br />
* Ecosystem partners (CDNs, encoding providers, landmark publishers, etc) are already getting used to ánd building tools around the concept of *chunked* video streams. Examples are log aggregators that roll up chunks servings into a single logline, or encoders that simultaneously build multiple stream levels, chunk them up and render their manifests.<br />
* With just the QOS metrics (*downloadRate* and *decodedFrames*) in place, it will be possible to build adaptive-streaming-like solutions (using range-requests) in javascript. In Flash, this same functionality is supported (and very popular) within both Flowplayer and JW Player. True adaptive streaming (continous switching without buffering) won't be possible, but the experience is good enough to suit people that don't have the encoder or browser (yet) to build or playback adaptive streams.<br />
<br />
[[Category:Proposals]]</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6357Video Metrics2011-04-09T13:52:32Z<p>Silviapfeiffer: added Silverlight stats</p>
<hr />
<div>'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''Silverlight''' [http://learn.iis.net/page.aspx/582/advanced-logging-for-iis-70---client-logging/ advanced logging] relevant stats:<br />
<br />
* c-buffercount: Number of times rebuffering is required. This is how many times we underflow basically. This is calculated at the frame level.<br />
* c-bytes: Number of bytes received by the client from the server. The value does not include any overhead that is added by the network stack. However, HTTP may introduce some overhead. Therefore, the same content streamed by using different protocols may result in different values. If c-bytes and sc-bytes(server-side) are not identical, packet loss occurred.<br />
* c-starttime: The point where the client began watching the stream (in seconds, no fraction). For true live streaming, we need to calculate time offset using wallclock time.<br />
* x-duration: Duration (in seconds) of the data rendered by the client from c-starttime.<br />
* startupTimeMs: From play to render first frame (in milliseconds).<br />
* bandwidthMax: Maximum perceived bandwidth<br />
* bandwidthMin: Minimum perceived bandwidth<br />
* bandwidthAvg: Average perceived bandwidth<br />
* droppedFramesPerSecond: Dropped frames per second (provided by Silverlight)<br />
* renderedFramesPerSecond: Rendered frames per second (provided by Silverlight)<br />
* audioResponseTimeAvg: Average response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMax: Maximum response time to get audio chunks. This is time from request to last byte.<br />
* audioResponseTimeMin: Minimum response time to get audio chunks. This is time from request to last byte.<br />
* videoResponseTimeAvg: Average response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMax: Maximum response time to get video chunks. This is time from request to last byte.<br />
* videoResponseTimeMin: Minimum response time to get video chunks. This is time from request to last byte.<br />
* audioDownloadErrors: Total number of missing audio chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs.<br />
* videoDownloadErrors: Total number of missing video chunks (for example, 404s). This is a semicolon-separated list of starttime/chunk IDs<br />
* audioPlaybackBitrates: An ordered list of the audio bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* videoPlaybackBitrates: An ordered list of the video bit-rates played during playback. This is a semicolon-separated list. This list is in the order of playback. There may be duplicate entries.<br />
* audioBandwidthAvg: Average audio bit rate for the downloaded chunks<br />
* videoBandwidthAvg: Average video bit rate for the downloaded chunks<br />
* audioBufferSizeAvg: Average audio buffer size (in seconds) during playback<br />
* audioBufferSizeMax: Maximum audio buffer size (in seconds) during playback<br />
* videoBufferSizeAvg: Average video buffer size (in seconds) during playback<br />
* videoBufferSizeMax: Maximum video buffer size (in seconds) during playback<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6355Video Metrics2011-04-07T06:08:41Z<p>Silviapfeiffer: improved intro</p>
<hr />
<div>'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
(1) One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
(2) Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
(3) A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Further note: Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occurred.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6354Video Metrics2011-04-07T06:02:12Z<p>Silviapfeiffer: added adobe flash stats</p>
<hr />
<div>'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination. Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
Service providers, especially commercial service providers, need to manage and monitor the performance of their service, for example to detect problems before they cause a deluge of Customer Service calls or identify whether, where and when the service is providing the quality of experience expected by customers. <br />
<br />
A related use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
Note that whenever bitrates are reported it needs to be clear how the bitrate is calculated. For example if it is an average then average over what time interval. If it is some kind of peak bitrate, then what's the window size over which the peak was calculated or other definition. A raw bitrate alone is not very meaningful.<br />
<br />
Measuring performance of media elements should include error cases. The HTMLMediaElement (and XmlHttpRequest for that matter) are rather light on reporting of network errors (just a boolean). It would be good to get more detailed information about what errors occured.<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''Adobe Flash player''' has [http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetStreamInfo.html#summaryTable many statistics]:<br />
<br />
* audioBufferByteLength - [read-only] Provides the NetStream audio buffer size in bytes.<br />
* audioBufferLength - [read-only] Provides NetStream audio buffer size in seconds.<br />
* audioByteCount- [read-only] Specifies the total number of audio bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* audioBytesPerSecond - [read-only] Specifies the rate at which the NetStream audio buffer is filled in bytes per second.<br />
* audioLossRate - [read-only] Specifies the audio loss for the NetStream session.<br />
* byteCount- [read-only] Specifies the total number of bytes that have arrived into the queue, regardless of how many have been played or flushed.<br />
* currentBytesPerSecond - [read-only] Specifies the rate at which the NetStream buffer is filled in bytes per second.<br />
* dataBufferByteLength - [read-only] Provides the NetStream data buffer size in bytes.<br />
* dataBufferLength - [read-only] Provides NetStream data buffer size in seconds.<br />
* dataByteCount - [read-only] Specifies the total number of bytes of data messages that have arrived in the queue, regardless of how many have been played or flushed.<br />
* dataBytesPerSecond - [read-only] Specifies the rate at which the NetStream data buffer is filled in bytes per second.<br />
* droppedFrames - [read-only] Returns the number of video frames dropped in the current NetStream playback session.<br />
* isLive -[read-only] Returns whether the media being played is recorded or live.<br />
* maxBytesPerSecond - [read-only] Specifies the maximum rate at which the NetStream buffer is filled in bytes per second.<br />
* metaData - [read-only] Retrieve last meta data object associated with media being played.<br />
* playbackBytesPerSecond - [read-only] Returns the stream playback rate in bytes per second.<br />
* SRTT - [read-only] The smoothed round trip time (SRTT) for the NetStream session, in milliseconds.<br />
* videoBufferByteLength - [read-only] Provides the NetStream video buffer size in bytes.<br />
* videoBufferLength - [read-only] Provides NetStream video buffer size in seconds.<br />
* videoByteCount - [read-only] Specifies the total number of video bytes that have arrived in the queue, regardless of how many have been played or flushed.<br />
* videoBytesPerSecond - [read-only] Specifies the rate at which the NetStream video buffer is filled in bytes per second.<br />
* videoLossRate - [read-only] Provides the NetStream video loss rate (ratio of lost messages to total messages).<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)<br />
<br />
<br />
'''MPEG DASH''' [http://mpeg.chiariglione.org/working_documents/mpeg-b/dash/dash-dis.zip defines] quality metrics for adaptive streaming at several levels<br />
* What is presented to the user i.e. which portions of which versions of the streams and when they were presented. This information implicitly includes within it information like startup delay, timing and duration of pauses due to buffer exhaustion and overall quality (since it includes when the rate adaptation changes happen)<br />
** This could be very simply represented as a sequence of ( Time, StreamId, Playback Rate ) tuples, one for every point in time at which the playback rate or stream changed<br />
* Buffer levels over time within the player<br />
* Performance of the network stack. This includes, for each HTTP request<br />
** The URL and byte range requested<br />
** The time when the request was sent, when the response started to arrive and when the response was completed<br />
** The amount of data received<br />
** HTTP response code and, if applicable, redirect URL<br />
** A more detailed trace of data arrival rate, for example bytes received in each 1s or 100ms interval<br />
<br />
All of this information is intended for performance monitoring purposes rather than to inform real-time action by the player. It's useful to separate these kinds of information. Performance monitoring information can be reported in batches to the application for reporting back to the server.<br />
<br />
HTTP performance information is sometimes collected at the server side. However, with adaptive streaming where streams are constructed at the client from many HTTP requests this becomes more difficult since HTTP requests for a single viewing session may be spread across multiple servers (or even multiple CDNs). It becomes more important as streaming services evolve to collect this information from the client.<br />
<br />
The DASH specification also includes information about the TCP level: what TCP connections were established and which HTTP requests were sent on which connection.<br />
<br />
The first kind of information above (what is presented) is almost already available based on video element events (changes in the current playback rate). The exception is rate adaptation changes.<br />
<br />
<br />
=== Network error codes ===<br />
<br />
It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6348Video Metrics2011-03-31T21:02:13Z<p>Silviapfeiffer: a bit of page formatting</p>
<hr />
<div>'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
== Requirements ==<br />
<br />
<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination. Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
A further use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
== Collection of Proposals/Implementations ==<br />
<br />
'''Mozilla''' have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded - number of audio bytes that have been decoded.<br />
* webkitVideoBytesDecoded - number of video bytes that have been decoded.<br />
* webkitDecodedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* webkitDroppedFrames - number of frames that were decoded but not displayed due to performance issues.<br />
<br />
<br />
'''JW Player''' (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS ''metrics'' for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels. Changes e.g. when the viewer jumps fullscreen.<br />
* qualityLevel: index of the currently playing quality level (see below).<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculated and broadcast upon completion of a fragment load.<br />
<br />
Separately, JW Player broadcasts a ''SWITCH'' event at the painting of a frame that has a different qualityLevel than the preceding frame(s). While the ''metrics.qualityLevel'' tells developers the qualityLevel of the currently downloading buffer/fragment, the ''SWITCH'' event tells developers the exact point in time where the viewer experiences a jump in video quality. This event also helps developers correlate the value of ''frameDropRate'' to the currently playing qualityLevel (as opposed to the currently loading one). Depending upon buffer, fragment and GOP size, the time delta between a change in ''metrics.qualityLevel'' and ''SWITCH.qualityLevel'' [http://www.cc.gatech.edu/~sakhshab/Saamer_MMSys11.pdf may vary] from a few seconds to a few minutes. <br />
<br />
Finally, JW Player [http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/16012/setting-up-rtmp-dynamic-streaming accepts and exposes] per video an array with quality levels (the distinct streams of a video between which the player can switch). For each quality level, properties like ''bitrate'', ''framerate'', ''height'' and ''width'' are available. The plain mapping using ''qualityLevel'' works b/c JW Player to date solely supports single A/V muxed dynamic/adaptive videos - no multi track. <br />
<br />
<br />
For '''HTTP adaptive streaming''' the [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics following statistics] have been proposed:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to '''expose playback rate statistic'''s has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics] that '''measure the QoS''' that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6342Video Metrics2011-03-31T11:45:42Z<p>Silviapfeiffer: added qos metrics</p>
<hr />
<div>'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
'''Requirements'''<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination. Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
A further use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
'''Collection of Proposals/Implementations'''<br />
<br />
Mozilla have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
Webkit have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded<br />
* webkitVideoBytesDecoded<br />
* webkitDecodedFrames<br />
* webkitDroppedFrames<br />
<br />
<br />
JW Player (using [http://help.adobe.com/nl_NL/AS3LCR/Flash_10.0/flash/net/NetStreamInfo.html actionscript]) broadcasts the following QOS metrics for both RTMP dynamic and HTTP adaptive:<br />
<br />
* bandwidth: server-client data rate, in kilobytespersecond.<br />
* latency: client-server-client roundtrip time, in milliseconds.<br />
* frameDropRate: number of frames not presented to the viewer, in frames per second.<br />
* screenWidth / screenHeight: dimensions of the video viewport, in pixels.<br />
* qualityLevel:<br />
<br />
Bandwidth and droprate are running metrics (averaged out). Latency and dimensions are sampled metrics (taken once). For RTMP dynamic, the metrics are broadcast at a settable interval (default 2s). For HTTP adaptive, metrics are calculate and broadcast for the last fragment that was loaded (for bandwidth/latency) or presented (for droprate/dimensions).<br />
<br />
<br />
Previously the following statistics have been proposed for [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics HTTP adaptive streaming]:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to expose playback rate statistics has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back<br />
<br />
<br />
[http://www.quora.com/Video-Analytics/What-are-the-most-valuable-metrics-in-online-video-analytics Here are a few metrics that measure the QoS] that a user receives:<br />
<br />
* playerLoadTime<br />
* streamBitrate<br />
(user interaction and playthrough can be measured using existing events)</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6339Video Metrics2011-03-31T04:31:42Z<p>Silviapfeiffer: added webkit stats</p>
<hr />
<div>'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
'''Requirements'''<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination. Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
A further use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
'''Collection of Proposals/Implementations'''<br />
<br />
Mozilla have implemented the following [http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html statistics into Firefox]:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
Webkit have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:<br />
<br />
* webkitAudioBytesDecoded<br />
* webkitVideoBytesDecoded<br />
* webkitDecodedFrames<br />
* webkitDroppedFrames<br />
<br />
<br />
Previously the following statistics have been proposed for [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics HTTP adaptive streaming]:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
<br />
Further, a requirement to expose playback rate statistics has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Video_Metrics&diff=6338Video Metrics2011-03-30T02:45:11Z<p>Silviapfeiffer: started the page ... not very far yet</p>
<hr />
<div><br />
'''Related HTML WG bug:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399<br />
<br />
<br />
'''Requirements'''<br />
For several reasons, we need to expose the performance of media elements to JavaScript.<br />
<br />
One concrete use case is that content publishers want to understand the quality of their content as being played back by their users and how much a user is actually playing back. For example, if a video always goes into buffering mode after 1 min for all users - maybe there is a problem in the encoding, or the video is too big for the typical bandwidth/CPU combination. Also, publishers want to track the metrics of how much of their video and audio files is actually being watched.<br />
<br />
A further use case is HTTP adaptive streaming, where an author wants to manually implement an algorithm for switching between different resources of different bandwidth or screen size. For example, if the user goes full screen and the user's machine and bandwidth allow for it, the author might want to switch to a higher resolution video.<br />
<br />
<br />
'''Collection of Proposals/Implementations'''<br />
<br />
Mozilla have implemented the following statistics into Firefox:<br />
<br />
* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.<br />
* mozDecodedFrames - number of frames that have been decoded - converted into YCbCr.<br />
* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".<br />
* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.<br />
* mozPaintDelay - the time delay between presenting the last frame and it being painted on screen (approximately).<br />
<br />
<br />
Previously the following statistics have been proposed for [http://wiki.whatwg.org/wiki/Adaptive_Streaming#QOS_Metrics HTTP adaptive streaming]:<br />
<br />
* downloadRate: The current server-client bandwidth (read-only).<br />
* videoBitrate: The current video bitrate (read-only).<br />
* droppedFrames: The total number of frames dropped for this playback session (read-only).<br />
* decodedFrames: The total number of frames decoded for this playback session (read-only).<br />
* height: The current height of the video element (already exists).<br />
* videoHeight: The current height of the videofile (already exists).<br />
* width: The current width of the video element (already exists).<br />
* videoWidth: The current width of the videofile (already exists). <br />
<br />
Further, a requirement to expose playback rate statistics has come out of [http://lists.w3.org/Archives/Public/public-html/2011Feb/0113.html issue-147]:<br />
<br />
* currentPlaybackRate: the rate at which the video/audio is currently playing back</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=5541Use cases for timed tracks rendered over video by the UA2010-10-29T23:03:57Z<p>Silviapfeiffer: added link to video examples</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
This example shows a case where animated positioning and angular change are highly recommended to achieve a desired style: http://www.youtube.com/watch?v=Qz2F-H7IHFM<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
Karaoke with color coded voice, where parts sung together have a unique color ([http://www.youtube.com/watch?v=5tOxfjHLK-A source]):<br />
<br />
<img src="http://bayimg.com/image/iamdlaaci.jpg" title="Jacky sings"><br />
<img src="http://bayimg.com/image/iamdpaaci.jpg" title="Landy sings"><br />
<img src="http://bayimg.com/image/jamdbaaci.jpg" title="Jacky+Landy sing"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
<br />
== Video Examples ==<br />
<br />
See http://www.dcmp.org/captioningkey/types_methods_styles.html and the pages following this one (click on the next topic at the bottom)</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_API-level_access_to_timed_tracks&diff=5522Use cases for API-level access to timed tracks2010-10-18T00:07:57Z<p>Silviapfeiffer: added another instance of chapter rendering</p>
<hr />
<div>This page contains examples of features that it is not expected we will require user agents to natively render in the first version, though it's likely that we will provide APIs sufficient to implement these features in JS.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Chapter Markers ==<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<img src="http://main.makeuseoflimited.netdna-cdn.com/wp-content/uploads/2008/02/hulu2.jpg"><br />
<img src="http://www.annodex.net/~silvia/chapter-screenshot.jpg"><br />
<br />
== Timed Annotations ==<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<img src="http://cache.gawkerassets.com/assets/images/17/2009/07/340x_bubbleply.jpg"><br />
<img src="http://www.vccafe.com/wp-content/uploads/2007/12/infoply.png"><br />
<br />
== Video Annotation ==<br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
* [http://digitalurban.blogspot.com/2008/11/video-tracking-and-manipulation-adobe.html Video Tracking and Manipulation] ([http://portal.acm.org/citation.cfm?id=1354647 A framework for video annotation, visualization, and interaction])<br />
** Floating name tags, speech bubbles, graffiti etc. following a timed path (e.g. a person moving around on the screen)<br />
** Annotating moving objects<br />
** Controlling video flow<br />
<br />
== Lyrics ==<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
== Linguistic Markup ==<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
== Linking ==<br />
<br />
<img src="http://www.widgetslab.com/wp-content/uploads/2008/06/image21.png"><br />
<img src="http://www.ustream.tv/blog/wp-content/uploads/2008/08/nahright-cap-hyperlink.jpg"><br />
<img src="http://www.stardestroyer.net/Mike/Movies/StarshipTroopers.jpg"><br />
<br />
* Linking rendered text to a URL<br />
* Linking entire cue to a URL<br />
* Linking overlaid graphics to a URL<br />
* Linking defined regions to a URL<br />
* Linking entire frame to a URL<br />
<br />
== User interfaces ==<br />
<img src="http://kylecrothers.com/images/articles/Kyle.OSD.theme.png"><br />
<img src="http://i.telegraph.co.uk/telegraph/multimedia/archive/01392/sky-green-button_1392969c.jpg"><br />
<img src="http://informitv.com/images/articles/echostar/CNN.jpg"><br />
<img src="http://junkyard.damowmow.com/429" title="multiple text tracks active at same time - here subtitles, chapters & linked slides"><br />
<br />
==Video Editing==<br />
<br />
<img src="http://www.laguidainformatica.it/wp-content/uploads/2008/08/eyespot2.png"><br />
<img src="http://www.techiequest.com/pic/jumpcut01.jpg"><br />
<img src="http://eresearch.griffith.edu.au/userfiles/audio_annotation_1.jpg"><br />
<img src="http://www.geniusdv.com/news_and_tutorials/2010/01/25/scrolling_text_video_tab.gif"><br />
<br />
== Content Selection ==<br />
<img src="http://www.topopage.net/wp-content/motionbox.jpg"></div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_API-level_access_to_timed_tracks&diff=4715Use cases for API-level access to timed tracks2010-04-25T13:30:07Z<p>Silviapfeiffer: added application UI</p>
<hr />
<div>This page contains examples of features that it is not expected we will require user agents to natively render in the first version, though it's likely that we will provide APIs sufficient to implement these features in JS.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Chapter Markers ==<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<img src="http://main.makeuseoflimited.netdna-cdn.com/wp-content/uploads/2008/02/hulu2.jpg"><br />
<br />
== Timed Annotations ==<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<img src="http://cache.gawkerassets.com/assets/images/17/2009/07/340x_bubbleply.jpg"><br />
<img src="http://www.vccafe.com/wp-content/uploads/2007/12/infoply.png"><br />
<br />
== Video Annotation ==<br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
* [http://digitalurban.blogspot.com/2008/11/video-tracking-and-manipulation-adobe.html Video Tracking and Manipulation] ([http://portal.acm.org/citation.cfm?id=1354647 A framework for video annotation, visualization, and interaction])<br />
** Floating name tags, speech bubbles, graffiti etc. following a timed path (e.g. a person moving around on the screen)<br />
** Annotating moving objects<br />
** Controlling video flow<br />
<br />
== Lyrics ==<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
== Linguistic Markup ==<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
== Linking ==<br />
<br />
<img src="http://www.widgetslab.com/wp-content/uploads/2008/06/image21.png"><br />
<img src="http://www.ustream.tv/blog/wp-content/uploads/2008/08/nahright-cap-hyperlink.jpg"><br />
<img src="http://www.stardestroyer.net/Mike/Movies/StarshipTroopers.jpg"><br />
<br />
* Linking rendered text to a URL<br />
* Linking entire cue to a URL<br />
* Linking overlaid graphics to a URL<br />
* Linking defined regions to a URL<br />
* Linking entire frame to a URL<br />
<br />
== User interfaces ==<br />
<img src="http://kylecrothers.com/images/articles/Kyle.OSD.theme.png"><br />
<img src="http://i.telegraph.co.uk/telegraph/multimedia/archive/01392/sky-green-button_1392969c.jpg"><br />
<img src="http://informitv.com/images/articles/echostar/CNN.jpg"><br />
<img src="http://junkyard.damowmow.com/429" title="multiple text tracks active at same time - here subtitles, chapters & linked slides"><br />
<br />
==Video Editing==<br />
<br />
<img src="http://www.laguidainformatica.it/wp-content/uploads/2008/08/eyespot2.png"><br />
<img src="http://www.techiequest.com/pic/jumpcut01.jpg"><br />
<img src="http://eresearch.griffith.edu.au/userfiles/audio_annotation_1.jpg"><br />
<img src="http://www.geniusdv.com/news_and_tutorials/2010/01/25/scrolling_text_video_tab.gif"><br />
<br />
== Content Selection ==<br />
<img src="http://www.topopage.net/wp-content/motionbox.jpg"></div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4714Use cases for timed tracks rendered over video by the UA2010-04-25T13:29:20Z<p>Silviapfeiffer: moving to other page</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_API-level_access_to_timed_tracks&diff=4713Use cases for API-level access to timed tracks2010-04-25T13:28:43Z<p>Silviapfeiffer: added starship troopers here</p>
<hr />
<div>This page contains examples of features that it is not expected we will require user agents to natively render in the first version, though it's likely that we will provide APIs sufficient to implement these features in JS.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Chapter Markers ==<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<img src="http://main.makeuseoflimited.netdna-cdn.com/wp-content/uploads/2008/02/hulu2.jpg"><br />
<br />
== Timed Annotations ==<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<img src="http://cache.gawkerassets.com/assets/images/17/2009/07/340x_bubbleply.jpg"><br />
<img src="http://www.vccafe.com/wp-content/uploads/2007/12/infoply.png"><br />
<br />
== Video Annotation ==<br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
* [http://digitalurban.blogspot.com/2008/11/video-tracking-and-manipulation-adobe.html Video Tracking and Manipulation] ([http://portal.acm.org/citation.cfm?id=1354647 A framework for video annotation, visualization, and interaction])<br />
** Floating name tags, speech bubbles, graffiti etc. following a timed path (e.g. a person moving around on the screen)<br />
** Annotating moving objects<br />
** Controlling video flow<br />
<br />
== Lyrics ==<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
== Linguistic Markup ==<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
== Linking ==<br />
<br />
<img src="http://www.widgetslab.com/wp-content/uploads/2008/06/image21.png"><br />
<img src="http://www.ustream.tv/blog/wp-content/uploads/2008/08/nahright-cap-hyperlink.jpg"><br />
<img src="http://www.stardestroyer.net/Mike/Movies/StarshipTroopers.jpg"><br />
<br />
* Linking rendered text to a URL<br />
* Linking entire cue to a URL<br />
* Linking overlaid graphics to a URL<br />
* Linking defined regions to a URL<br />
* Linking entire frame to a URL<br />
<br />
== User interfaces ==<br />
<img src="http://kylecrothers.com/images/articles/Kyle.OSD.theme.png"><br />
<img src="http://i.telegraph.co.uk/telegraph/multimedia/archive/01392/sky-green-button_1392969c.jpg"><br />
<img src="http://informitv.com/images/articles/echostar/CNN.jpg"><br />
<br />
==Video Editing==<br />
<br />
<img src="http://www.laguidainformatica.it/wp-content/uploads/2008/08/eyespot2.png"><br />
<img src="http://www.techiequest.com/pic/jumpcut01.jpg"><br />
<img src="http://eresearch.griffith.edu.au/userfiles/audio_annotation_1.jpg"><br />
<img src="http://www.geniusdv.com/news_and_tutorials/2010/01/25/scrolling_text_video_tab.gif"><br />
<br />
== Content Selection ==<br />
<img src="http://www.topopage.net/wp-content/motionbox.jpg"></div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4712Use cases for timed tracks rendered over video by the UA2010-04-25T13:27:20Z<p>Silviapfeiffer: moving to other page</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Example of multiple text tracks active at same time and displayed ===<br />
<br />
This one has subtitles, chapters, and through events linked slides.<br />
<br />
<img src="http://junkyard.damowmow.com/429"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4711Use cases for timed tracks rendered over video by the UA2010-04-24T23:55:00Z<p>Silviapfeiffer: added example with multiple tracks active at the same time</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Hyperlinks on caption cues or individual words ===<br />
<br />
Still need some examples from the real world. Here's one from a science fiction movie:<br />
<br />
<img src="http://www.stardestroyer.net/Mike/Movies/StarshipTroopers.jpg"><br />
<br />
=== Example of multiple text tracks active at same time and displayed ===<br />
<br />
This one has subtitles, chapters, and through events linked slides.<br />
<br />
<img src="http://junkyard.damowmow.com/429"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4709Use cases for timed tracks rendered over video by the UA2010-04-24T09:10:03Z<p>Silviapfeiffer: No, this is not a joke - I think it will be very important for monetisation of timed text tracks!</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Hyperlinks on caption cues or indivdual words ===<br />
<br />
<img src="http://www.stardestroyer.net/Mike/Movies/StarshipTroopers.jpg"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4708Use cases for timed tracks rendered over video by the UA2010-04-24T09:02:51Z<p>Silviapfeiffer: added examples with color</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<img src="http://dashiva.net/misc/vertical_karaoke.jpg"><br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
This example shows that we need line-based positioning: http://www.youtube.com/watch?v=bw5JBWdaUHI#t=0m12s<br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<img src="http://dashiva.net/misc/bouncing_karaoke.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<img src="http://dashiva.net/misc/multiple_speakers.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Multiple colors ===<br />
<br />
Colors are often used in closing comments to signify the author of the captions.<br />
<br />
<img src="http://junkyard.damowmow.com/426"><br />
<img src="http://junkyard.damowmow.com/428"><br />
<br />
<br />
=== Legal notices ===<br />
<img src="http://fifty-strength.com/images/freestuff/backgrounds/COPYRIGHT_WARNING_blk.gif"><br />
<br />
=== Watermarks ===<br />
<img src="http://img62.imageshack.us/img62/1985/natgeowild01.jpg"><br />
<br />
=== Informative ===<br />
<img src="http://www.dmst.aueb.gr/dds/pubs/jrnl/2004-CG-ITV/html/CS04_files/image002.jpg"><br />
<img src="http://www.julyseventh.co.uk/images/sky-news-aldgate-east-fatalities.jpg"><br />
<img src="http://informitv.com/images/articles/redbee/Piero-02.jpg"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4680Use cases for timed tracks rendered over video by the UA2010-04-22T23:33:15Z<p>Silviapfeiffer: /* Multiple voices */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/BCCS_unbelievably_wordy_speaker_ID_1.jpg" title="provides speaker names"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4679Use cases for timed tracks rendered over video by the UA2010-04-22T23:30:43Z<p>Silviapfeiffer: /* Positioning */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<img src="http://joeclark.org/access/crtc/CRTC-2008/reply/images/CCfoto-CBC-CC-ST-ManWithoutaFace.jpg" title="things gone wrong"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4676Use cases for timed tracks rendered over video by the UA2010-04-22T23:17:59Z<p>Silviapfeiffer: /* Background emphasis */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/425"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4675Use cases for timed tracks rendered over video by the UA2010-04-22T23:17:31Z<p>Silviapfeiffer: added background emphasis example</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Background emphasis ===<br />
<br />
This is from a mobile phone and needs particular contrast and visibility.<br />
<br />
<img src="http://junkyard.damowmow.com/424"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4674Use cases for timed tracks rendered over video by the UA2010-04-22T23:15:02Z<p>Silviapfeiffer: /* Positioning */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<img src="http://junkyard.damowmow.com/424" title="placed under speaker"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4672Use cases for timed tracks rendered over video by the UA2010-04-22T23:13:57Z<p>Silviapfeiffer: /* Positioning */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<img src="http://junkyard.damowmow.com/423" title="placed under speaker"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4671Use cases for timed tracks rendered over video by the UA2010-04-22T23:12:27Z<p>Silviapfeiffer: /* Plain text */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<img src="http://junkyard.damowmow.com/422"><br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4670Use cases for timed tracks rendered over video by the UA2010-04-22T23:10:55Z<p>Silviapfeiffer: /* Italics or other inline emphasis */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<img src="http://junkyard.damowmow.com/421" alt="italics and blod used together"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Use_cases_for_timed_tracks_rendered_over_video_by_the_UA&diff=4640Use cases for timed tracks rendered over video by the UA2010-04-21T02:31:13Z<p>Silviapfeiffer: /* Positioning */</p>
<hr />
<div>This page contains examples of features where timed tracks are rendered over video by user agents natively.<br />
<br />
Return to [[timed tracks]] home page.<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<img src="http://img3.imageshack.us/img3/6743/vlcsnap090208083027134xq5.png"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Italics or other inline emphasis ===<br />
<img src="http://dashiva.net/misc/1271498287684.jpg"><br />
<img src="http://daed.site24.se/Screenshot20100420222809422.jpg"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<img src="http://hsivonen.iki.fi/screen/subtitles.jpg"><br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415" title="(uses unicode U+266B ♫ or U+266A ♪♪)">]<br />
<br />
=== Formal documentation for real-world captioning ===<br />
<br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual] requires:<br />
** minor control over position<br />
** word-level italics control<br />
** frame-level timing<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4587Timed tracks2010-04-20T04:14:07Z<p>Silviapfeiffer: /* Timed Annotations */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<img src="http://cache.gawkerassets.com/assets/images/17/2009/07/340x_bubbleply.jpg"><br />
<img src="http://www.vccafe.com/wp-content/uploads/2007/12/infoply.png"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Some Background Reading ====<br />
<br />
* [http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
* [http://www.masternewmedia.org/video_internet_television/video-annotation/online-video-annotation-takes-a-giant-leap-20070405.htm most varied experiments with time-aligned "text" by company Mojiti (now out of biz)]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* color of background/text/outline is needed for readability on different types of video.<br />
* webfonts is needed to provide high quality subtitles in some non-Latin languages (e.g. Chinese where a suitable font is unlikely to be available even on Chinese computer systems).<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4585Timed tracks2010-04-20T04:08:17Z<p>Silviapfeiffer: /* Timed Annotations */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<img src="http://cache.gawkerassets.com/assets/images/17/2009/07/340x_bubbleply.jpg"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Some Background Reading ====<br />
<br />
* [http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
* [http://www.masternewmedia.org/video_internet_television/video-annotation/online-video-annotation-takes-a-giant-leap-20070405.htm most varied experiments with time-aligned "text" by company Mojiti (now out of biz)]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4584Timed tracks2010-04-20T04:07:41Z<p>Silviapfeiffer: /* Lyrics */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Some Background Reading ====<br />
<br />
* [http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
* [http://www.masternewmedia.org/video_internet_television/video-annotation/online-video-annotation-takes-a-giant-leap-20070405.htm most varied experiments with time-aligned "text" by company Mojiti (now out of biz)]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4583Timed tracks2010-04-20T04:00:24Z<p>Silviapfeiffer: /* Subtitle/Caption/Karaoke File Format */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Some Background Reading ====<br />
<br />
* [http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
* [http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
* [http://www.masternewmedia.org/video_internet_television/video-annotation/online-video-annotation-takes-a-giant-leap-20070405.htm most varied experiments with time-aligned "text" by company Mojiti (now out of biz)]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4582Timed tracks2010-04-20T03:54:19Z<p>Silviapfeiffer: /* Timed Annotations */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
[http://blog.makezine.com/archive/2008/06/youtube_has_onvideo_annot.html <img src="http://junkyard.damowmow.com/417">]<br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4581Timed tracks2010-04-20T03:49:56Z<p>Silviapfeiffer: /* Chapter Markers */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<img src="http://www.veotag.com/img/whatis/screen.jpg"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4580Timed tracks2010-04-20T03:46:59Z<p>Silviapfeiffer: added lyrics examples</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<img src="http://pcwin.com/media/images/screen/68930-singalong_player.jpg"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<img src="http://farm4.static.flickr.com/3107/2375923033_ee4309922a.jpg"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Lyrics ===<br />
<br />
[http://svg-wow.org/audio/animated-lyrics.html animated Lyrics in HTML5]<br />
<img src="http://svg-wow.org/audio/animated-lyrics_poster.png"><br />
<img src="http://www.mobile-mir.com/images/LyricShow.png"><br />
<img src="http://lh4.ggpht.com/itswaps/R2Qmlc9KnMI/AAAAAAAAAb8/GrPzd1MChQc/4_minilyrics1%5B6%5D"><br />
<img src="http://images.intomobile.com/wp-content/uploads/2008/01/iphone-lyrics-display-karaoke-mode.jpg" title="seems to be an iphone app with video with lyrics"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4579Timed tracks2010-04-20T03:21:42Z<p>Silviapfeiffer: added linguistic markup examples</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Linguistic Markup ===<br />
<br />
<img src="http://www.ling.canterbury.ac.nz/jen/onzeminer/PraatTextGrid.jpg"><br />
<img src="http://junkyard.damowmow.com/416"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4578Timed tracks2010-04-20T02:59:05Z<p>Silviapfeiffer: /* Ticker Text */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<img src="http://img.youtube.com/vi/pWe7wTVbLUU/0.jpg"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4577Timed tracks2010-04-20T02:55:15Z<p>Silviapfeiffer: added ticker text example</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<br />
=== Ticker Text ===<br />
<br />
<img src="http://media.rgemonitor.com/images/blogs/yahoo_finance_tech_ticker_6_18_09_1.jpg"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4576Timed tracks2010-04-20T02:52:44Z<p>Silviapfeiffer: added timed annotations</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Timed Annotations ===<br />
<br />
<img src="http://www.techsmith.com/img/learn/camtasia/6/custom-callouts/callout_customex.jpg"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4575Timed tracks2010-04-20T02:30:10Z<p>Silviapfeiffer: added musical notes example</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Musical Notes ===<br />
<br />
[http://www.screencast.com/users/dro/folders/Jing/media/628d4d8f-f96c-4444-b853-a27645cf6ebf <img src="http://junkyard.damowmow.com/415">]<br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4574Timed tracks2010-04-20T02:20:21Z<p>Silviapfeiffer: /* Subtitle/Caption/Karaoke File Format */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Musical notes in the subtitles<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
[http://joeclark.org/access/captioning/CBC/images/CBC-captioning-manual-EN.pdf CBC captioning manual]<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4573Timed tracks2010-04-20T02:03:17Z<p>Silviapfeiffer: /* Subtitle/Caption/Karaoke File Format */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Musical notes in the subtitles<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
[http://main.wgbh.org/wgbh/pages/mag/services/captioning/faq/sugg-styles-conv-faq.html Rules given to professional captioners] <br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4572Timed tracks2010-04-20T01:52:34Z<p>Silviapfeiffer: /* Chapter Markers */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
[http://www.cmivfx.com <img src="http://junkyard.damowmow.com/414">]<br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Musical notes in the subtitles<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4571Timed tracks2010-04-20T01:50:28Z<p>Silviapfeiffer: /* Chapter Markers */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
<a href="http://www.cmivfx.com"><img src="http://junkyard.damowmow.com/414"></a><br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Musical notes in the subtitles<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeifferhttps://wiki.whatwg.org/index.php?title=Timed_tracks&diff=4570Timed tracks2010-04-20T01:49:29Z<p>Silviapfeiffer: /* Chapter Markers */</p>
<hr />
<div>I'm starting to look at the feedback sent over the past few years for<br />
augmenting audio and video with additional timed tracks such as subtitles,<br />
captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing<br />
that would be really helpful is if we could get together a representative<br />
sample of typical uses of these features, as well as examples of some of<br />
the more extreme uses.<br />
<br />
If anyone has any examples, please add them below. <br />
Links to either videos or stills showing subtitles (e.g. on TVs, DVDs,<br />
etc) are both good.<br />
I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —[[User:Hixie|Hixie]] 21:07, 16 April 2010 (UTC)<br />
<br />
<br />
== Static Examples ==<br />
<br />
These are categorised by what features they demonstrate that is most interesting.<br />
<br />
=== Positioning ===<br />
<br />
<img src="http://philip.html5.org/misc/eva-captions.jpg"><br />
<br />
=== Ruby ===<br />
<br />
<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"><br />
<img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke"><br />
<br />
=== Karaoke ===<br />
<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"><br />
<img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"><br />
<img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines"><br />
<br />
=== Multiple voices ===<br />
<br />
<img src="http://philip.html5.org/misc/portal-caption.jpg"><br />
<br />
=== Important line feeds ===<br />
<br />
<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"><br />
<img src="http://joshkinberg.com/blog/files/debatewars.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"><br />
<img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"><br />
<img src="http://damowmow.com/temp/mpc-storm-codec.JPG"><br />
<br />
=== Plain text ===<br />
<br />
<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"><br />
<img src="http://dolphy-tech.net/files/subtitles_mkv.png"><br />
<img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"><br />
<img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"><br />
<img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"><br />
<img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"><br />
<img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"><br />
<img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"><br />
<img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"><br />
<img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"><br />
<img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"><br />
<img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"><br />
<img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"><br />
<img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"><br />
<img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"><br />
<img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"><br />
<img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"><br />
<img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif"><br />
<br />
=== Chapter Markers ===<br />
<br />
<img src="http://images.apple.com/quicktime/tutorials/images/screenshot-chaptertrack1.jpg"><br />
<img src="http://www.arguingwithmyself.com/wordpress/wp-content/uploads/ted-video-player.png"><br />
<img src="http://junkyard.damowmow.com/414"><br />
<img src="http://www.davemeyerson.com/img/port/big/flash/VideoPlayer_01.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/11_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.kenstone6.net/fcp_homepage/images_hdv_to_prores_with_chapter_markers_stone/13_hdv_prores_chapter_markers_stone.jpg"><br />
<img src="http://www.componentone.com/newimages/Products/ScreenShots/StudioWPF/C1MediaPlayer/MediaPlayer_ChapterList.png"><br />
<img src="http://maccreate.com/uploads/2009/06/200906162256.jpg"><br />
<img src="http://simplemoviex.com/SimpleMovieX/documentation/handrake,%20itunes,%20metadata.picChapter.jpeg"><br />
<img src="http://www.js8media.com/moviechapterizer/files/page3_3.png"><br />
<img src="http://www.macupdate.com/images/screens/uploaded/29332_sm.png"><br />
<br />
=== Features for which we have no examples ===<br />
<br />
(Are these myths?)<br />
<br />
* Musical notes in the subtitles<br />
* Individual words being emphasised<br />
* More complicated inline formatting<br />
<br />
<br />
== Dynamic Examples ==<br />
<br />
=== Automatic text placement ===<br />
<br />
This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40<br />
<br />
== Requirements ==<br />
<br />
=== Subtitle/Caption/Karaoke File Format ===<br />
<br />
==== Structure ====<br />
<br />
* multiple voices<br />
* per-segment time in/out cues<br />
* inline time cues for karaoke<br />
* bidi, newlines, ruby<br />
* position<br />
<br />
==== Positioning ====<br />
<br />
* vertical: top/middle/bottom/% (default bottom)<br />
* horizontal: left/center/right/% (default center)<br />
* display modes: replace previous text, scroll previous text up and add to bottom<br />
* multiple voices placed in adjacent places would need to automatically stack so they don't overlap<br />
* multiple segments with overlapping times would need to be stacked so they don't overlap<br />
<br />
(Percentage positions would work like background-position in CSS.)<br />
<br />
==== Formatting ====<br />
<br />
* no font or colour formatting seems to be needed<br />
* no inline emphasis seems to be needed<br />
* support for bidi is critical<br />
* some cases use ruby<br />
* providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)<br />
<br />
=== Audio Descriptions File Format ===<br />
<br />
* Just timing and text?<br />
<br />
=== HTML ===<br />
<br />
* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them<br />
<br />
* format for external subtitles/captions<br />
* format for external audio descriptions<br />
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio descriptions<br />
* an API to allow a segment to be dynamically inserted into the rendering on the fly<br />
<br />
* an API for exposing what the currently relevant segments of each timed track are<br />
* a way to hook into this mechanism to advance slides<br />
<br />
* native rendering of subtitles<br />
* native rendering of audio descriptions<br />
* native rendering of multiple audio or video tracks, to allow pre-recorded audio descriptions to be mixed in and sign language video to be overlaid<br />
* a way to hook into this to manually render timed tracks</div>Silviapfeiffer