Video Metrics - Revision history

LloydW: Update webkit metric property names to correct values (see https://trac.webkit.org/browser/trunk/ManualTests/video-statistics.html)

2015-03-05T11:36:19Z

Update webkit metric property names to correct values (see https://trac.webkit.org/browser/trunk/ManualTests/video-statistics.html)

← Older revision		Revision as of 11:36, 5 March 2015
Line 34:		Line 34:
	'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:		'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:

	* ~~webkitAudioBytesDecoded~~ - number of audio bytes that have been decoded.		* webkitAudioDecodedByteCount - number of audio bytes that have been decoded.
	* ~~webkitVideoBytesDecoded~~ - number of video bytes that have been decoded.		* webkitVideoDecodedByteCount - number of video bytes that have been decoded.
	* ~~webkitDecodedFrames~~ - number of frames that have been demuxed and extracted out of the media.		* webkitDecodedFrameCount - number of frames that have been demuxed and extracted out of the media.
	* ~~webkitDroppedFrames~~ - number of frames that were decoded but not displayed due to performance issues.		* webkitDroppedFrameCount - number of frames that were decoded but not displayed due to performance issues.

Silviapfeiffer: added moz bug link

2012-05-03T23:49:22Z

added moz bug link

← Older revision		Revision as of 23:49, 3 May 2012
Line 30:		Line 30:
	* mozFrameDelay - the time delay between presenting the last frame and it being painted on screen (approximately).		* mozFrameDelay - the time delay between presenting the last frame and it being painted on screen (approximately).

			Mozilla are also [https://bugzilla.mozilla.org/show_bug.cgi?id=686370 working on some of the statistics listed here].

	'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:		'''Webkit''' have [https://bugs.webkit.org/show_bug.cgi?id=53322 implemented] these:

Silviapfeiffer: updating Firefox stats

2012-05-03T23:48:12Z

updating Firefox stats

← Older revision		Revision as of 23:48, 3 May 2012
Line 22:		Line 22:
	== Collection of Proposals/Implementations ==		== Collection of Proposals/Implementations ==

	'''Mozilla''' have implemented the following [~~http~~://~~blog~~.~~pearce~~.org~~.nz~~/~~2011~~/03/~~html5-video-painting-performance.html~~ statistics into Firefox]:		'''Mozilla''' have [https://bugzilla.mozilla.org/show_bug.cgi?id=580531 implemented] the following [https://developer.mozilla.org/en/DOM/HTMLVideoElement statistics into Firefox]:

	* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.		* mozParsedFrames - number of frames that have been demuxed and extracted out of the media.
Line 28:		Line 28:
	* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".		* mozPresentedFrames - number of frames that have been presented to the rendering pipeline for rendering - were "set as the current image".
	* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.		* mozPaintedFrames - number of frames which were presented to the rendering pipeline and ended up being painted on the screen. Note that if the video is not on screen (e.g. in another tab or scrolled off screen), this counter will not increase.
	* ~~mozPaintDelay~~ - the time delay between presenting the last frame and it being painted on screen (approximately).		* mozFrameDelay - the time delay between presenting the last frame and it being painted on screen (approximately).


Line 157:		Line 157:

	It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.		It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.


	== Proposal ==		== Proposal ==

Silviapfeiffer: added use cases for metrics

2011-10-11T04:03:21Z

added use cases for metrics

← Older revision		Revision as of 04:03, 11 October 2011
Line 170:		Line 170:
	Applicable to: <video>, <audio>		Applicable to: <video>, <audio>

	The raw bytes received from the network for decode. ~~This~~ can be used to calculate the effective bandwidth. Includes container data.		The raw bytes received from the network for decode. Together with the downloadTime, this can be used to calculate the effective bandwidth. Includes container data.

			'''Use case:''' This measures the network performance. This would be used to report the effective video bandwidth received by the browser to the server (i.e. the content publisher) to better understand what bandwidth is typically available to users and encode the videos on the server in the right set of bitrates (e.g. for what bandwidths are typically received in mobile / in desktop environments in different countries, so what should the edge servers be seeded with).

			===downloadTime===
			Applicable to: <video>, <audio>

			The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).

			'''Use case:''' This allows to determine network performance together with "bytesReceived" by providing what time has actually been spent for downloading the resource (parts). This is better than just starting a timer when putting the resource URL into the video element, since times where the browser does not try to download more parts of the resource are not part of this downloadTime.


			===networkWaitTime===
			Applicable to: <video>, <audio>

			The total duration of time when a playback is stalled, waiting to receive more data from network.

			'''Use case:''' This allows to determine how much of the downloadTime is actually waiting time and thus allows to determine situations of network congestion and delays and separate that from the bandwidth that is available to the user.


	===videoBytesDecoded===		===videoBytesDecoded===
Line 178:		Line 196:

	See [[#bytesdecodedsuggestion\|suggestion]] below for a proposed alternative form.		See [[#bytesdecodedsuggestion\|suggestion]] below for a proposed alternative form.

			'''Use case:''' This measures the decoding pipeline performance. This would be used to report whether the decoding pipeline keeps up with the playback position, given that the network feeds sufficient bytes into this pipeline. videoBytesDecoded would measure the performance of the video decoding pipeline.


	===audioBytesDecoded===		===audioBytesDecoded===
Line 183:		Line 204:

	The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.		The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.

			'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded. audioBytesDecoded would measure the performance of the audio decoding pipeline.

	:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').		:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').

			'''Use case:''' This measures the decoding pipeline performance similar to videoBytesDecoded and audioBytesDecoded, but would be applicable per track (and is therefore probably more generally applicable). It can e.g. help identify slow browsers, or plugins that slow down the decoding, or the optimal number of tracks to put into a file before the decoding engine gets overloaded.


	===decodedFrames===		===decodedFrames===
Line 190:		Line 216:

	The number of frames of video that have been decoded and made available for presentation.		The number of frames of video that have been decoded and made available for presentation.

			'''Use case:''' This measure the decoding pipeline performance on a frame level rather than a byte level to allow making judgements about the rendering engine when comparing to droppedFrames.


	===droppedFrames===		===droppedFrames===
Line 195:		Line 224:

	The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.		The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.

			'''Use case:''' This measures the number of frames that are already too late to be handed to the rendering pipeline after decoding and thus measures the speed of the decoding pipeline.


	===presentedFrames===		===presentedFrames===
Line 201:		Line 233:
	The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.		The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.

	~~===downloadTime===~~		'''Use case:''' This measures the display quality that the user perceives and can determine the performance of the rendering engine given the performance of the network and decoding pipeline. If, for example, the system receives sufficient data from the network (no droppedFrames), but the rate of presented frames per second is below 15, we can assume that the user gets a pretty crappy presentation because the rendering engine was too slow, the machine is likely overloaded or not capable of rendering this video at this quality and we should probably move to a lower bitrate (resolution or framerate) resource for the video.
	~~Applicable to~~:~~<video>~~,~~<audio>~~
	~~The time since first HTTP request~~ is ~~sent until now~~ or ~~the download stops/finishes/terminates~~ (~~whichever is earlier~~).

	~~===networkWaitTime===~~
	~~The total duration of time when a playback is stalled, waiting to receive more data from network.~~

	===playbackJitter===		===playbackJitter===

Silviapfeiffer: added metworkWaitTime

2011-10-05T01:44:12Z

added metworkWaitTime

← Older revision		Revision as of 01:44, 5 October 2011
Line 204:		Line 204:
	Applicable to:<video>,<audio>		Applicable to:<video>,<audio>
	The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).		The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).

			===networkWaitTime===
			The total duration of time when a playback is stalled, waiting to receive more data from network.

	===playbackJitter===		===playbackJitter===

Silviapfeiffer: added downloadTime

2011-10-05T01:40:01Z

added downloadTime

← Older revision		Revision as of 01:40, 5 October 2011
Line 1:		Line 1:
	'''Related HTML WG ~~bug~~:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399		'''Related HTML WG bugs:''' http://www.w3.org/Bugs/Public/show_bug.cgi?id=12399 , http://www.w3.org/Bugs/Public/show_bug.cgi?id=14310


Line 200:		Line 200:

	The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.		The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.

			===downloadTime===
			Applicable to:<video>,<audio>
			The time since first HTTP request is sent until now or the download stops/finishes/terminates (whichever is earlier).

	===playbackJitter===		===playbackJitter===

Steve: /* Proposal */

2011-05-06T04:44:25Z

Proposal

← Older revision		Revision as of 04:44, 6 May 2011
Line 168:		Line 168:

	===bytesReceived===		===bytesReceived===
	Applicable to: <video>, <audio>.		Applicable to: <video>, <audio>

	The raw bytes received from the network for decode. This can be used to calculate the effective bandwidth. Includes container data.		The raw bytes received from the network for decode. This can be used to calculate the effective bandwidth. Includes container data.
Line 180:		Line 180:

	===audioBytesDecoded===		===audioBytesDecoded===
	Applicable to: <audio>, <video>.		Applicable to: <audio>, <video>

	The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.		The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.
Line 187:		Line 187:

	===decodedFrames===		===decodedFrames===
	Applicable to: <video>.		Applicable to: <video>

	The number of frames of video that have been decoded and made available for presentation.		The number of frames of video that have been decoded and made available for presentation.

	===droppedFrames===		===droppedFrames===
	Applicable to: <video>.		Applicable to: <video>

	The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.		The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.

	===presentedFrames===		===presentedFrames===
	Applicable to: <video>.		Applicable to: <video>

	The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.		The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.

	===playbackJitter===		===playbackJitter===
	Applicable to: <video>.		Applicable to: <video>

	It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:		It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:

Steve: /* playbackJitter */ Clarify what 'expected number of frames is'.

2011-05-06T04:43:33Z

playbackJitter: Clarify what 'expected number of frames is'.

← Older revision		Revision as of 04:43, 6 May 2011
Line 213:		Line 213:
	:playbackJitter = sum(abs(Ei - Ai))		:playbackJitter = sum(abs(Ei - Ai))

	The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered.		The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered (which would normally be presentedFrames + droppedFrames).

	[[Category:Proposals]]		[[Category:Proposals]]

Steve at 06:47, 5 May 2011

2011-05-05T06:47:28Z

← Older revision		Revision as of 06:47, 5 May 2011
Line 214:		Line 214:

	The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered.		The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered.

			[[Category:Proposals]]

Steve: Added a proposal.

2011-05-02T20:48:02Z

Added a proposal.

← Older revision		Revision as of 20:48, 2 May 2011
Line 157:		Line 157:

	It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.		It's difficult to define an exhaustive, implementation-independent list of errors. A common solution is to report an "error chain" which is a sequence of increasingly-specific error codes, each of which is the "cause" of the more general error preceding it in the chain. At the end of the chain, implementation-specific error codes can be used. Simple applications can interpret the earlier, standardized, high-level errors. Commercial applications may have an incentive to interpret some of the implementation-specific ones - at least the ones they see often.


			== Proposal ==

			The attributes specified here are intended to contain only those that are not calculable by the application from any other source (or be derivable from themselves). E.g. there is no need for a ‘frames per second’ attribute.

			To be more specific, there are no time based derivatives as any sampling window will never be applicable for all applications - an application developer can use the attribute value to calculate the incoming rate by creating a timer and calculating the difference in values in successive timer calls.

			''Open issue: Do these values reset when the source media has changed?''

			===bytesReceived===
			Applicable to: <video>, <audio>.

			The raw bytes received from the network for decode. This can be used to calculate the effective bandwidth. Includes container data.

			===videoBytesDecoded===
			Applicable to: <video>

			The number of bytes of video data that have been decoded. This can be used to calculate the effective video bitrate. Does not include container data.

			See [[#bytesdecodedsuggestion\|suggestion]] below for a proposed alternative form.

			===audioBytesDecoded===
			Applicable to: <audio>, <video>.

			The number of bytes of audio data that have been decoded. This can be used to calculate the effective audio bitrate. Does not include container data.

			:<span id="bytesdecodedsuggestion"></span>'''Suggestion''': Rename '''videoBytesDecoded''' and '''audioBytesDecoded''' to a simple '''bytesDecoded''' which is the sum of all decoded audio and video data (excluding container data) across all constituent tracks and potentially also expose '''bytesDecoded''' at the TrackList level (i.e. add '''getBytesDecoded(in unsigned long index)''').

			===decodedFrames===
			Applicable to: <video>.

			The number of frames of video that have been decoded and made available for presentation.

			===droppedFrames===
			Applicable to: <video>.

			The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.

			===presentedFrames===
			Applicable to: <video>.

			The number of frames of video that have been presented for compositing into the page. This could be used to calculate frames per second.

			===playbackJitter===
			Applicable to: <video>.

			It is useful for applications to be able obtain an overall metric for perceived playback quality and smoothness. This value is the sum of all duration errors for frames intended to be presented to the user, where:

			; Ei : Desired duration of frame i spent on the screen (to nearest microsecond?)
			; Ai : Actual duration frame i spent on the screen (if the frame is never presented to the user, then Ai == 0).

			then:

			:playbackJitter = sum(abs(Ei - Ai))

			The application could use this value by sampling it at the beginning and end of a five second window and dividing it by the expected number of frames delivered.