Timed tracks

I'm starting to look at the feedback sent over the past few years for augmenting audio and video with additional timed tracks such as subtitles, captions, audio descriptions, karaoke, slides, lyrics, ads, etc. One thing that would be really helpful is if we could get together a representative sample of typical uses of these features, as well as examples of some of the more extreme uses.

If anyone has any examples, please add them below. Links to either videos or stills showing subtitles (e.g. on TVs, DVDs, etc) are both good. I'd like to get a representative sample so that we can deteremine what features are critical, and what features can be punted for now. —Hixie 21:07, 16 April 2010 (UTC)

Static Examples

These are categorised by what features they demonstrate that is most interesting.

Positioning

<img src="http://philip.html5.org/misc/eva-captions.jpg">

Ruby

<img src="http://graphics8.nytimes.com/images/blogs/screens/06subs.jpg"> <img src="http://4.bp.blogspot.com/_JDuyiEFkBc0/R5wEOwaqiBI/AAAAAAAAAhQ/08eKZHlPR2k/s400/yako13.jpg" title="also shows karaoke">

Karaoke

<img src="http://i.ytimg.com/vi/5ZbQMq6q9yw/0.jpg"> <img src="http://img.youtube.com/vi/xMPJfStmcJY/0.jpg" title="also shows multiple voices"> <img src="http://i2.ytimg.com/vi/dTMAiX0ShuM/0.jpg" title="also shows newlines">

Multiple voices

<img src="http://philip.html5.org/misc/portal-caption.jpg">

Important line feeds

<img src="http://www.craphound.com/images/itcrowdleetsubs.jpg"> <img src="http://joshkinberg.com/blog/files/debatewars.jpg"> <img src="http://208.71.113.236/final/2/8/2831661/358818.jpg"> <img src="http://cdn.fd.uproxx.com/wp-content/uploads/2009/03/letheright1_or.jpg"> <img src="http://damowmow.com/temp/mpc-storm-codec.JPG">

Plain text

<img src="http://www.insidesocal.com/tomhoffarth/Miss-Teen-South-Carolina-Subtitles.jpg"> <img src="http://dolphy-tech.net/files/subtitles_mkv.png"> <img src="http://filegets.com/screenshots/full/subtitle-player_15496.jpg"> <img src="http://www.bbc.co.uk/blogs/bbcinternet/img/iplayer_subtitles_russia.jpg"> <img src="http://startupmeme.com/wp-content/uploads/2008/08/youtubecaptions-thumb1.png"> <img src="http://1.bp.blogspot.com/_EuCTzLdp3vE/SYtn40aQjDI/AAAAAAAACXY/95EgLfOttiA/s400/video_captions.jpg"> <img src="http://208.71.113.236/final/2/8/2852791/405302.jpg"> <img src="http://jeanviet.info/astimg/sous-titre-divx.jpg"> <img src="http://www.mobiletopsoft.com/images/news/tcpmp_gora_playback_subtitles_2.jpg"> <img src="http://www.tiresias.org/research/guidelines/television/images/subtitles2.jpg"> <img src="http://img117.exs.cx/img117/5200/AnimalPlanetTV.jpg"> <img src="http://duhn.net/wp-content/uploads/dtt-tv2-subtitles.jpg"> <img src="http://www.hack7mc.com/wp-content/uploads/2009/04/earthstoodstill.png"> <img src="http://img.youtube.com/vi/nqGOOTjxTZ0/0.jpg"> <img src="http://aboutonlinetips.com/wp-content/uploads/2008/12/subtitles-in-movie.jpg"> <img src="http://thepinksylphide.com/images/compare/cardcaptor-sakura-dvds/AnimeCartoon-Subtitle-1.png"> <img src="http://bbsimg.ngfiles.com/1/17187000/ngbbs48a5483851c0d.jpg"> <img src="http://i.msdn.microsoft.com/ms971317.atg_ccandaudiodesc_01(en-us,MSDN.10).gif">

Features for which we have no examples

(Are these myths?)

• Musical notes in the subtitles
• Individual words being emphasised
• More complicated inline formatting

Dynamic Examples

Automatic text placement

This example shows how multiple segments with overlapping times should be automatically placed so as to not overlap: http://www.youtube.com/watch?v=xG9KluukpJI#t=2m40

Requirements

Subtitle/Caption/Karaoke File Format

Structure

• multiple voices
• per-segment time in/out cues
• inline time cues for karaoke
• bidi, newlines, ruby
• position

Positioning

• vertical: top/middle/bottom/% (default bottom)
• horizontal: left/center/right/% (default center)
• display modes: replace previous text, scroll previous text up and add to bottom
• multiple voices placed in adjacent places would need to automatically stack so they don't overlap
• multiple segments with overlapping times would need to be stacked so they don't overlap

(Percentage positions would work like background-position in CSS.)

Formatting

• no font or colour formatting seems to be needed
• no inline emphasis seems to be needed
• support for bidi is critical
• some cases use ruby
• providing a pseudo-element to style each voice would likely be sufficient for authors who want overall formatting control (this would also allow user overrides conveniently)

Audio Descriptions File Format

• Just timing and text?

HTML

• native rendering of subtitles
• subtitle format for external subtitles
• some mechanism for text in the page to be grabbed for titles
• an abstraction of subtitle formats
• an API for exposing what the current titles are
• an API to allow a title to be dynamically inserted
• a way to hook into this mechanism to advance slides
• a way to hook into this to manually render the titles