Faster-than-Realtime DASH Stream: Incorrectly Calculated End Time

[rhyswilliamsza]

What version of Shaka Player are you using?

• v3.2.8+
• v3.3.8+
• v4.0.4+
• v4.1.2+
• v4.2.0+

Can you reproduce the issue with our latest release version?
Yes

Can you reproduce the issue with the latest code from main?
Yes

Are you using the demo app or your own custom app?
Custom App

If custom app, can you reproduce the issue using our demo app?
Yes

What browser and OS are you using?
All Browsers

For embedded devices (smart TVs, etc.), what model and firmware version are you using?
N/A

What are the manifest and license server URIs?
Any DASH/HLS manifest that grows faster than realtime. Difficult to provide a static manifest as an example.

What configuration are you using? What is the output of player.getConfiguration()?

Config

``` { "drm": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "servers": {}, "clearKeys": {}, "advanced": {}, "delayLicenseRequestUntilPlayed": false, "logLicenseExchange": false, "updateExpirationTime": 1, "preferredKeySystems": [] }, "manifest": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "availabilityWindowOverride": null, "disableAudio": false, "disableVideo": false, "disableText": false, "disableThumbnails": false, "defaultPresentationDelay": 0, "dash": { "clockSyncUri": "", "ignoreDrmInfo": false, "disableXlinkProcessing": false, "xlinkFailGracefully": false, "ignoreMinBufferTime": false, "autoCorrectDrift": true, "initialSegmentLimit": 1000, "ignoreSuggestedPresentationDelay": false, "ignoreEmptyAdaptationSet": false, "ignoreMaxSegmentDuration": false, "keySystemsByURI": { "urn:uuid:1077efec-c0b2-4d02-ace3-3c1e52e2fb4b": "org.w3.clearkey", "urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed": "com.widevine.alpha", "urn:uuid:9a04f079-9840-4286-ab92-e65be0885f95": "com.microsoft.playready", "urn:uuid:79f0049a-4098-8642-ab92-e65be0885f95": "com.microsoft.playready", "urn:uuid:f239e769-efa3-4850-9c16-a903c6932efb": "com.adobe.primetime" } }, "hls": { "ignoreTextStreamFailures": false, "ignoreImageStreamFailures": false, "useFullSegmentsForStartTime": false, "defaultAudioCodec": "mp4a.40.2", "defaultVideoCodec": "avc1.42E01E" } }, "streaming": { "retryParameters": { "maxAttempts": 2, "baseDelay": 1000, "backoffFactor": 2, "fuzzFactor": 0.5, "timeout": 30000, "stallTimeout": 5000, "connectionTimeout": 10000 }, "rebufferingGoal": 2, "bufferingGoal": 300, "bufferBehind": 30, "ignoreTextStreamFailures": false, "alwaysStreamText": false, "startAtSegmentBoundary": false, "gapDetectionThreshold": 0.1, "smallGapLimit": 0.5, "jumpLargeGaps": false, "durationBackoff": 1, "forceTransmuxTS": false, "safeSeekOffset": 5, "stallEnabled": true, "stallThreshold": 1, "stallSkip": 0.1, "useNativeHlsOnSafari": true, "inaccurateManifestTolerance": 2, "lowLatencyMode": false, "autoLowLatencyMode": false, "forceHTTPS": false, "preferNativeHls": false, "updateIntervalSeconds": 1, "dispatchAllEmsgBoxes": false, "observeQualityChanges": false }, "offline": { "usePersistentLicense": true }, "abr": { "enabled": true, "useNetworkInformation": true, "defaultBandwidthEstimate": 1000000, "switchInterval": 8, "bandwidthUpgradeTarget": 0.85, "bandwidthDowngradeTarget": 0.95, "restrictions": { "minWidth": 0, "maxWidth": null, "minHeight": 0, "maxHeight": null, "minPixels": 0, "maxPixels": null, "minFrameRate": 0, "maxFrameRate": null, "minBandwidth": 0, "maxBandwidth": null }, "advanced": { "minTotalBytes": 128000, "minBytes": 16000, "fastHalfLife": 2, "slowHalfLife": 5 } }, "preferredAudioLanguage": "", "preferredTextLanguage": "", "preferredVariantRole": "", "preferredTextRole": "", "preferredAudioChannelCount": 2, "preferredVideoCodecs": [], "preferredAudioCodecs": [], "preferForcedSubs": false, "preferredDecodingAttributes": [], "restrictions": { "minWidth": 0, "maxWidth": null, "minHeight": 0, "maxHeight": null, "minPixels": 0, "maxPixels": null, "minFrameRate": 0, "maxFrameRate": null, "minBandwidth": 0, "maxBandwidth": null }, "playRangeStart": 0, "playRangeEnd": null, "cmcd": { "enabled": false, "sessionId": "", "contentId": "", "useHeaders": false } } ```

What did you do?

1. Start playing dash manifest that grows faster than real-time (i.e. on-demand encode).
2. Witness that 'player.seekRange().end' grows by 1s every second, instead of using the total duration of available segments.

What did you expect to happen?

1. Start playing dash manifest that grows faster than real-time (i.e. on demand encode).
2. The endTime should grow to display the full available data set in the dash manifest.

What actually happened?
PR #4348, fixing an unrelated bug, introduced a static presentationStartTime_ in presentation_timeline.js, resulting in the presentationStartTime_ remaining locked for the duration of play.

This presentationStartTime_ variable is calculated to be now - this.maxSegmentEndTime_ - this.maxSegmentDuration_. Once the new start time variable becomes locked, this calculation is skipped. Hence, we now only calculate this value once.

If maxSegmentEndTime_ grows as per a live stream, i.e. 1s per second, this would not be an issue. The presentationStartTime_ would remain static, and the function to calculate the live edge (now - this.maxSegmentDuration_ - this.presentationStartTime_), would work as expected. However, if maxSegmentEndTime_ grows by more than 1s per second, the newly available segments are lost.

e.g. when Shaka begins playing the file, ffmpeg has encoded 3 minutes worth of footage. Example, now = 500.

now = 500
maxSegmentEndTime_ = 3 * 60 = 180
maxSegmentDuration_ = e.g. 5
presentationStartTime_ = 500 - 180 - 5 = 315
liveEdge = 500 - (500 - 180 - 5)  - 5
liveEdge = 180

As it a dynamic manifest, Shaka will continue to fetch the mpd until it moves to a static type. If ffmpeg continues encoding faster than real-time, and now has 5 minutes worth of footage encoded 10 seconds later, Shaka now calculates the following.

now = 510
maxSegmentEndTime_ = 5 * 60  = 300
maxSegmentDuration_ = e.g. 5
presentationStartTime_ = LOCKED at 315.
liveEdge = 510 - (315) - 5
liveEdge = 190

In the above example, we see that although the manifest has 300 seconds of footage listed, we still receive a liveEdge time that moves 1s per second.

Ideally, we would have calculated the below:

now = 510
maxSegmentEndTime_ = 5 * 60  = 300
maxSegmentDuration_ = e.g. 5
presentationStartTime_ = 500 - 300 - 5 = 195.
liveEdge = 510 - (195) - 5
liveEdge = 310

This is not necessarily a fault of the above PR, but is likely just a result of using presentation_timeline.js in a peculiar way. Please let me know if this is intended behaviour, and I will fork & roll out a custom version of shaka for my use-case. Thanks!

[avelad]

@joeyparrish can you review it? Thanks!

[stevenewing]

Hey @avelad! What was the behaviour you were seeing in the browser as a result of these calculations?