WHATWG Wiki - User contributions [en]

Talk:Timed tracks

2010-06-10T19:10:03Z

Joeclark: /* Contrary to Hixie’s claim, I didn’t make any “mistakes” */ new section

== This will not end well ==

You’re making more than the usual number of mistakes and, given Hixie’s imperviousness to any evidence he doesn’t agree with, your spec will without question end up being so flawed it cannot and will not be used.

== Wheel reinvention? ==

Don't we already have the [http://www.w3.org/TR/ttaf1-dfxp/ Timed Text] spec for this sort of thing?

== Contrary to Hixie’s claim, I didn’t make any “mistakes” ==

My edits to this spec are based on 30 years’ experience in captioning, which is 30 years more than Hixie has. He knows next to nothing about the topic and his edits, which he glossed as “mistakes” (18:52, 21 April 2010), obliterate known use cases – known, that is, to actual captioning viewers.

I say again: This will not end well. It’s another example of Hixie’s petty tyranny. The only upside is that petty is the only tyranny he can manage.

Talk:Timed tracks

2010-04-21T18:26:29Z

Joeclark: Created page with '== This will not end well == You’re making more than the usual number of mistakes and, given Hixie’s imperviousness to any evidence he doesn’t agree with, your spec will w...'

Timed tracks

2010-04-21T18:25:22Z

Joeclark: Fixing merely the most obvious errors. Fixing the whole list would take all week

This page contains notes for the development of the first version of timed track features in HTML.

See also [[use cases for timed tracks rendered over video by the UA]], [[use cases for API-level access to timed tracks]].

== Requirements ==

=== Subtitle/Caption/Karaoke File Format ===

==== Structure ====

* multiple rendered titles
* per-segment time in/out cues
* inline time cues for karaoke
* bidi, newlines, ruby
* full typographic features
* positioning

==== Positioning ====

* vertical: top/middle/bottom/absolute/relative (default bottom)
* horizontal: left/center/right/absolute/relative (default center for subtitles only)
* text alignment: horizontal text: left, right, center, at tab stop; vertical text: top, middle, bottom, at tab stop
* display modes: replace previous text, add to previous text, scroll previous text up and add to bottom, paint text on to create single block, clear screen

Multiple voices placed in adjacent places would need to automatically stack so they don't overlap. Multiple segments with overlapping times would need to be stacked so they don't overlap.

(Relative positions could work like background-position in CSS.)

==== Typography ====

===== Inline =====
* horizontal and vertical text directions, with changes of same, must be supported
* some cases use ruby
* italics and typeface changes (and, rarely, underlining and small caps) necessary

===== Global =====
* color (foreground, background mask, drop shadow, outline, other) is needed for readability on different types of video
* cannot ship without native ability to create and alter background mask, which is not the largest rectangle that can be drawn around all displayed text
* Webfonts desirable
* providing a classname to style each voice could likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)

=== Audio Description ===

Cannot work from the assumption that we are providing only a script for a computer to read out loud, a use case never attempted in the real world. Actual audio descriptions are prerecorded voices mixed into main audio. Redefining the spec to pretend that a script for a computer to read aloud constitutes “audio description” stands to be rejected by actual blind users.

=== Dubbing ===

If our spec defines audio description (added audio track), it must also define dubbing (another added audio track).

=== Combinations ===

All combinations of features must be possible and no combination must be banned. Common use cases include:

* dubbing with captioning
* dubbing with subtitling
* captioning with subtitling
* audio description with dubbing
* audio description with captioning
* audio description with subtitling

=== HTML ===

* an API and UI for exposing what timed tracks exist and selectively enabling/disabling them

* format for external subtitles/captions
* format for external audio description
* some mechanism for text in the page to be used instead of external files, for subtitles/captions or audio description
* an API to allow a segment to be dynamically inserted into the rendering on the fly

* an API for exposing what the currently relevant segments of each timed track are
* a way to hook into this mechanism to advance slides

* native rendering of subtitles
* native rendering of audio description
* native rendering of multiple audio or video tracks, to allow pre-recorded audio description to be mixed in and sign language video to be overlaid
* a way to hook into this to manually render timed tracks

== Architecture ==

<img src="http://docs.google.com/drawings/pub?id=1GR6Pzq0GY2n1sx_ZjDfuICM2LnXxLVxzvyl4kuQy-48&w=640&h=480">

Somebody needs to write an alt text for this image.

=== Declaring timed tracks ===

Each timed track is either:

*enabled, in which case it is downloaded, triggers events, and if appropriate is rendered by the user agent; or
*disabled, in which case it does nothing

The enabled/disabled state is by default based on user preferences and the kind of timed track as described below, but can be overridden on a per-track basis.

Each timed track has a kind which is one of:
* for visual display (subtitles, captions, translations), enabled based on user preferences, shows in video playback area
* for audio playback (audio description, dubbing), enabled based on user preferences, renders as audio
* for navigation (chapter titles), enabled by default, shows in UA UI
* for off-video display (lyrics), disabled by default in this version, not shown by UA
* for metadata (slide timings, annotation data for app-rendered annotations), enabled by default, not shown by UA

Tracks that are for visual display or audio playback have additionally a user-facing label and a language.

Tracks that are for visual display have an additional boolean indicating if they include sound effects and speaker identification (intended for the deaf, hard of hearing, or people with sound muted) or not (i.e. translations intended for people with audio enabled but who cannot understand the language, or karaoke lyrics).

Each timed track associated with a media resource, like the media resource itself, can have multiple sources.

Each source for a timed track has:
* URL
* type (if there are multiple sources)
* media

The media resource can also imply certain timed tracks based on data in the media resource.

The script can also add “virtual” timed tracks dynamically.

=== CSS extensions for styling captions ===

[TK]

=== DOM API ===

[TK]

=== Other minor things ===

We need to make sure that media playback is paused until all enabled timed tracks are locally available.

== Open issues ==

* How do we handle sign-language tracks?
* Do we handle multiple alternate audio tracks from this or is that restricted to in-band data? (in the media resource)

Talk:HTML vs. XHTML

2006-12-05T05:07:21Z

Joeclark: table inside p? I think not

An often repeated assertion is that XHTML is as different from HTML as RDF/XML is from N3. And that the proper way to tell the two apart is via MIME types.

There are only two problems with that. XHTML is not as different from HTML as RDF/XML is from N3. And MIME types can't be relied on. Let's take each in turn.

=== Syntax ===

* Both N3 and RDF/XML are used to express sets of RDF triples. They are equally capable: every triple store can be dumped into either format. The analogy here is the DOM. It is not currently the case that every DOM tree can be dumped equally capably into either format.
* N3 and RDF/XML are not the same, nor do they even look similar. They are different from top to bottom. Not only are no N3 documents valid RDF/XML, there are no individual triples that can be expressed the same way in both formats.
** You need to explain how RDF/N3 is relevant! --[[User:Lachlan Hunt|Lachlan Hunt]] 04:43, 4 December 2006 (UTC)
*** The top of this page starts with "An often repeated assertion is that XHTML is as different from HTML as RDF/XML is from N3". Need I provide references? -[[User:Rubys|Rubys]] 14:08, 4 December 2006 (UTC)

=== Mime Types ===

* People have consistently proven that they can't be trusted to configure and set MIME types correctly. Most aren't even aware that MIME types exist. The default setup with Apache is to not allow overrides. One popular use case is for documentation that is served via <code>file:///</code> URIs directly from your hard disk.
** <code>file:///</code> URIs use an OS or browser specific mechanism to determine the MIME. On Windows, for instance (for IE), the file extension is mapped to a MIME type via a key in the registry. --[[User:Lachlan Hunt|Lachlan Hunt]] 11:16, 4 December 2006 (UTC)
*** and as such, can rarely be depended upon. In addition to file extensions, content sniffing is also a common strategy. -[[User:Rubys|Rubys]] 14:12, 4 December 2006 (UTC)
* HTTP as specified indicates that the the <code>Content-Type</code> header is authoritative - it trumps the XML prolog. HTTP as practiced treats the MIME type as a hint. Whether it be feeds or WMV files, users have an expectation as to what happens when they click on these links, and are unhappy when the browser lets them down.
** For compatibility, those issues with several file formats do, unfortunately, have to be retained. However, breaking Content-Type in that way for <code>text/html</code> to somehow allow the content to be treated as XML instead is not an option. --[[User:Lachlan Hunt|Lachlan Hunt]] 11:16, 4 December 2006 (UTC)

It isn't clear to me (Hixie), however, how the fact that authors can't set the MIME type properly is supposed to be something we can ever solve from the point of view of the syntax of HTML. The full XML syntax isn't compatible with HTML parsers, and the full HTML syntax isn't compatible with XML parsers. The common subset is a tiny language that doesn't support widely used features like <style> or scripting. We can't parse text/html files as anything but HTML. The parser used for content sent with XML MIME types is out of scope for the WHATWG specs (it would be up to the XML guys). It isn't that we WANT the MIME type to be the only way to distinguish the two. It's that the MIME type IS the only way. It's a statement of fact, not desire. [[User:Hixie|Hixie]] 18:24, 4 December 2006 (UTC)
* [http://planet.intertwingly.net/ my planet] is served as <code>application/xhtml+xml</code> to Firefox and <code>text/html</code> to IE. It seems to be capable of doing both scripting and style in both modes. -[[User:Rubys|Rubys]]

=== Ideals ===

In an ideal word:
* the syntax of XML and HTML would be either complete identical or completely different.
** The syntax of HTML and XHTML are completely different. The fact that they look similar on the surface is irrelevant. (see above). --[[User:Lachlan Hunt|Lachlan Hunt]] 04:43, 4 December 2006 (UTC)
*** Completely? I'd say that they are as different as en-us and en-au. :-) -[[User:Rubys|Rubys]]
* the set of DOM trees that could be serialized as XHTML and HTML would either be completely identical or completely different.
** This is not possible without breaking backwards compatibility. These incompatibilities have existed between HTML and XHTML for a long time, and that hasn't stopped people serialising their XHTML as HTML up until now (for all practical purposes, serving XHTML as text/html is equivalent to reserialising). --[[User:Lachlan Hunt|Lachlan Hunt]] 11:16, 4 December 2006 (UTC)
* <code>Content-Type</code> would either always be respected, or always be ignored.
** <code>Content-Type</code> is always respected for for HTML and XHTML MIME types. It's not for some others, but that's a different issue --[[User:Lachlan Hunt|Lachlan Hunt]] 11:16, 4 December 2006 (UTC)
*** Always? Try serving your feed as text/html to FireFox 2.0. -[[User:Rubys|Rubys]]
*** Try serving your feed as text/html to *any browser* with feed support. [[User:Sayrer|Sayrer]]
* there would either be a fool-proof way to "sniff" whether the a given content was HTML or XHTML; or there would be no difference between XHTML and HTML in terms of syntax and range of DOM trees that could validly be serialized would also be identical.
** There is a foolproof way... the MIME type. :-) -Hixie

=== Analysis ===

Obviously, the current situation is less than ideal. XML and HTML evolved from a common ancestor. XML isn't changing. And the constraint to be as backwards compatible with HTML4 as humanly possible places practical limits on what can be done. Neither being absolutely identical with the XML syntax nor being completely different are options.

At the present time, the HTML5 syntax is a (near) superset of the XHTML syntax. Yet the situation is (nearly) reversed for the set of DOM trees that can be serialized into XHTML is larger than the set of DOM trees that can be serialized into HTML5.

Having the syntaxes being substantially similar leads to confusion in some edge cases (e.g., <code><p/></code>) but also has some advantages. Similar syntaxes would make things easier for people who have become disillusioned with XHTML and wish to migrate to HTML5. Conversely, similar syntaxes would make incremental migration from HTML5 to XHTML5 easier for those who wish to take advantage of the greater set of DOM trees that can be represented in that syntax.

=== Potential Strategies ===

'''Note''': these strategies are not necessarily mutually-exclusive.

* Develop better tools and actively work to integrate them into products like WordPress and DreamWeaver. (We're doing this already. -Hixie)
* The definition of HTML5 understandably and correctly puts a higher weight on HTML4 compatibility than XHTML migration. But as a migration aid, identify some unlikely/invalid combination (example: use of the HTML5 DOCTYPE combined with <code>xmlns</code> attribute on the <code>html</code> element combined with the use of a non-xml MIME type) and adjust some (as of yet undefined) set of the HTML5 parsing rules.
* Document these differences, either in the spec itself (as a non-normative appendix?) and/or by having a conformance checker flag these differences. Variations:
** Ensure that each of these differences triggers a [http://www.whatwg.org/specs/web-apps/current-work/#parse parse error] or equivalent in HTML5; this does not (necessarily) involve changing the recovery action or the way the document is ultimately parsed.
** Instead of bothering people who may not care about these differences, identify some unlikely combination (such as the DOCTYPE/xmlns/MIME combination above) and have it trigger a '''pedantic mode''' which enables these additional checks.

=== table inside p? ===

Do you really mean the following?

<blockquote>
In XHTML, p elements may contain structured inline level elements including blockquote, dl, menu, ol, ul, pre and table
</blockquote>

In what respect are blockquote, dl, menu, ol, ul, and table “inline,” and how are they allowed inside p?
– [[User:Joeclark|Joeclark]] 05:07, 5 December 2006 (UTC)