A user account is required in order to edit this wiki, but we've had to disable public user registrations due to spam.

To request an account, ask an autoconfirmed user on IRC (such as one of these permanent autoconfirmed members).

Difference between revisions of "Video Overlay"

From WHATWG Wiki
Jump to: navigation, search
(References)
(work in progress...)
Line 1: Line 1:
<overlay> is a proposal to display and style subtitles/captions for <video> in a uniform way, regardless of whether they are in-band or from an external resource. It also doubles as a container to overlay arbitrary (HTML) content on a <video>, enabling advanced scripted overlays such as karaoke animations or visual annotations.
+
&lt;overlay> is a proposal to display and style subtitles/captions for &lt;video> in a uniform way, regardless of whether they are in-band or from an external resource. It also doubles as a container to overlay arbitrary (HTML) content on a &lt;video>, enabling advanced scripted overlays such as karaoke animations or visual annotations.
 
 
== Background ==
 
 
 
  
 
== Use Case Description ==
 
== Use Case Description ==
''Complete description of the use case.''
+
There are several distinct use cases addressed by this proposal:
 +
* Linking &lt; with external captions/subtitles for native fetching/decoding/display by the UA.
 +
* Styling captions/subtitles with CSS, regardless of their source.
 +
* Allowing scripts to operate on captions/subtitles in a uniform manner, regardless of their source.
 +
''Possible sources of captions/subtitles include in-band (e.g. embedded in an MPEG-4 or Ogg stream), external (e.g. SRT or DXFP) or scripted (e.g. extracted from an on-page transcript) captions/subtitles.''
  
 
=== Current Limitations ===
 
=== Current Limitations ===
HTML5 currently lacks convenient markup and interfaces to handle at least three things:
+
HTML5 currently lacks convenient markup and/or interfaces to handle at least these things:
* Syncing and displaying external subtitles/captions with <video>
+
* Syncing and styling external subtitles/captions with &lt;video>
 
* Styling in-band subtitles/captions from media resources
 
* Styling in-band subtitles/captions from media resources
*  
+
* Rendering scripted controls on top of &lt;video> and positioning them to bottom.
* Placing
+
* Callbacks at specific times for scripted subtitles/captions (previously possible with "cue ranges")
 +
* Allowing any overlay (controls/captions/etc) to be retained in fullscreen mode.
  
 
=== Current Usage and Workarounds ===
 
=== Current Usage and Workarounds ===
''Some evidence that this feature is desperately needed on the webYou may provide a separate examples page for listing these.''
+
Currently no browser supports rendering in-band subtitles, so there are no workarounds for styling them. Fullscreen support is still immature, but there is no possible workaround for having scripted captions or controls to appear in fullscreen display.
 +
 
 +
==== Scripted Captions ====
 +
In Silvia's [http://www.annodex.net/~silvia/itext/elephant_no_skin_v2.html &lt;itext> demo] external SRT subtitles are fetched with XHR, parsed with JavaScript and finally synced in the timeupdate event. Using the timeupdate event is sub-optimal because it isn't guaranteed to fire any more often than every 250 ms, which isn't enough for fast-paced dialog.
 +
 
 +
==== Scripted Controls ====
 +
In order to overlay scripted controls on top of &lt;video>, a wrapping &lt;div> and some CSS is needed:
 +
&lt;div style="position:relative;width:400px;height:300px">
 +
  &lt;video src="video.ogv" style="width:100%;height:100%">&lt;/video>
 +
  &lt;div class="controls" style="position:absolute;bottom:0;left:0;right:0">
 +
  &lt;!-- actual controls here -->
 +
  &lt;/div>
 +
  &lt;/div>
 +
This isn't terrible, but requires the size of the video to be known or be fixed to a certain size as above.
  
 
=== Benefits ===
 
=== Benefits ===
Line 22: Line 37:
  
 
=== Requests for this Feature ===
 
=== Requests for this Feature ===
 +
* &lt;overlay> [http://lists.w3.org/Archives/Public/public-html-a11y/2009Nov/0098.html Suggested] by Philip Jägenstedt (Opera)
  
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote>
+
''TODO: Find the many mails related to some of the features addressed by &lt;overlay>''
  
 
== Proposed Solutions ==
 
== Proposed Solutions ==
Line 50: Line 66:
 
* [http://blog.gingertech.net/2009/11/23/model-of-a-time-linear-media-resource/ The model of a time-linear media resource for HTML5]
 
* [http://blog.gingertech.net/2009/11/23/model-of-a-time-linear-media-resource/ The model of a time-linear media resource for HTML5]
 
* [http://blog.gingertech.net/2009/11/25/manifests-exposing-structure-of-a-composite-media-resource/ Manifests for exposing the structure of a Composite Media Resource]
 
* [http://blog.gingertech.net/2009/11/25/manifests-exposing-structure-of-a-composite-media-resource/ Manifests for exposing the structure of a Composite Media Resource]
Silvia Pfeiffer's <itext> proposals:
+
Silvia Pfeiffer's &lt;itext> proposals:
 
* [https://wiki.mozilla.org/Accessibility/HTML5_captions HTML5 captions] ([https://wiki.mozilla.org/Accessibility/Experiment1_feedback feedback])
 
* [https://wiki.mozilla.org/Accessibility/HTML5_captions HTML5 captions] ([https://wiki.mozilla.org/Accessibility/Experiment1_feedback feedback])
 
* [https://wiki.mozilla.org/Accessibility/HTML5_captions_v2 HTML5 captions v2]
 
* [https://wiki.mozilla.org/Accessibility/HTML5_captions_v2 HTML5 captions v2]
Line 56: Line 72:
 
* [http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-May/014887.html re-thinking "cue ranges"] from David Singer (Apple)
 
* [http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-May/014887.html re-thinking "cue ranges"] from David Singer (Apple)
 
* [http://lists.w3.org/Archives/Public/public-html-a11y/2009Nov/0089.html timing model of the media resource in HTML5] from Silvia Pfeiffer
 
* [http://lists.w3.org/Archives/Public/public-html-a11y/2009Nov/0089.html timing model of the media resource in HTML5] from Silvia Pfeiffer
* [http://lists.w3.org/Archives/Public/public-html-a11y/2009Nov/0098.html initial <overlay> suggestion] from Philip Jägenstedt (Opera) with feedback from Eric Carlson (Apple)
+
* [http://lists.w3.org/Archives/Public/public-html-a11y/2009Nov/0098.html initial &lt;overlay> suggestion] from Philip Jägenstedt (Opera) with feedback from Eric Carlson (Apple)
  
 
[[Category:Feature Request|empty Template]]
 
[[Category:Feature Request|empty Template]]

Revision as of 12:05, 28 November 2009

<overlay> is a proposal to display and style subtitles/captions for <video> in a uniform way, regardless of whether they are in-band or from an external resource. It also doubles as a container to overlay arbitrary (HTML) content on a <video>, enabling advanced scripted overlays such as karaoke animations or visual annotations.

Use Case Description

There are several distinct use cases addressed by this proposal:

  • Linking < with external captions/subtitles for native fetching/decoding/display by the UA.
  • Styling captions/subtitles with CSS, regardless of their source.
  • Allowing scripts to operate on captions/subtitles in a uniform manner, regardless of their source.

Possible sources of captions/subtitles include in-band (e.g. embedded in an MPEG-4 or Ogg stream), external (e.g. SRT or DXFP) or scripted (e.g. extracted from an on-page transcript) captions/subtitles.

Current Limitations

HTML5 currently lacks convenient markup and/or interfaces to handle at least these things:

  • Syncing and styling external subtitles/captions with <video>
  • Styling in-band subtitles/captions from media resources
  • Rendering scripted controls on top of <video> and positioning them to bottom.
  • Callbacks at specific times for scripted subtitles/captions (previously possible with "cue ranges")
  • Allowing any overlay (controls/captions/etc) to be retained in fullscreen mode.

Current Usage and Workarounds

Currently no browser supports rendering in-band subtitles, so there are no workarounds for styling them. Fullscreen support is still immature, but there is no possible workaround for having scripted captions or controls to appear in fullscreen display.

Scripted Captions

In Silvia's <itext> demo external SRT subtitles are fetched with XHR, parsed with JavaScript and finally synced in the timeupdate event. Using the timeupdate event is sub-optimal because it isn't guaranteed to fire any more often than every 250 ms, which isn't enough for fast-paced dialog.

Scripted Controls

In order to overlay scripted controls on top of <video>, a wrapping <div> and some CSS is needed:

<div style="position:relative;width:400px;height:300px">
 <video src="video.ogv" style="width:100%;height:100%"></video>
 <div class="controls" style="position:absolute;bottom:0;left:0;right:0">
  <!-- actual controls here -->
 </div>
</div>

This isn't terrible, but requires the size of the video to be known or be fixed to a certain size as above.

Benefits

Explanation of how and why new markup would be useful.

Requests for this Feature

  • <overlay> Suggested by Philip Jägenstedt (Opera)

TODO: Find the many mails related to some of the features addressed by <overlay>

Proposed Solutions

My Solution

Brief description of the solution and of how it address the problem at hand.

Processing Model

Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.

Limitations

Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.

Implementation

Description of how and why browser vendors would take advantage of this feature.

Adoption

Reasons why page authors would use this solution.

References

Silvia Pfeiffer's blog posts:

Silvia Pfeiffer's <itext> proposals:

Mailing lists: