Skip to content

Commit

Permalink
docs: add diagrams for playlist loaders (#426)
Browse files Browse the repository at this point in the history
  • Loading branch information
ldayananda authored and gkatsev committed Mar 21, 2019
1 parent 7040b7d commit 52201f9
Show file tree
Hide file tree
Showing 38 changed files with 734 additions and 51 deletions.
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ This project supports both [HLS][hls] and [MPEG-DASH][dash] playback in the vide

A standard HLS stream consists of a *Master Playlist* which references one or more *Media Playlists*. Each Media Playlist contains one or more sequential video segments. All these components form a logical hierarchy that informs the player of the different quality levels of the video available and how to address the individual segments of video at each of those levels:

![HLS Format](hls-format.png)
![HLS Format](images/hls-format.png)

HLS streams can be delivered in two different modes: a "static" mode for videos that can be played back from any point, often referred to as video-on-demand (VOD); or a "live" mode where later portions of the video become available as time goes by. In the static mode, the Master and Media playlists are fixed. The player is guaranteed that the set of video segments referenced by those playlists will not change over time.

Expand Down
2 changes: 1 addition & 1 deletion docs/arch.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This project has three primary duties:
### Playlist Management
The [playlist loader](../src/playlist-loader.js) handles all of the details of requesting, parsing, updating, and switching playlists at runtime. It's operation is described by this state diagram:

![Playlist Loader States](playlist-loader-states.png)
![Playlist Loader States](images/playlist-loader-states.png)

During VOD playback, the loader will move quickly to the HAVE_METADATA state and then stay there unless a quality switch request sends it to SWITCHING_MEDIA while it fetches an alternate playlist. The loader enters the HAVE_CURRENT_METADATA when a live stream is detected and it's time to refresh the current media playlist to find out about new video segments.

Expand Down
8 changes: 4 additions & 4 deletions docs/bitrate-switching.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,24 @@ below. Whenever a new segment is downloaded, we calculate the download
bitrate based on the size of the segment and the time it took to
download:

![New bitrate info is available](bitrate-switching-1.png)
![New bitrate info is available](images/bitrate-switching-1.png)

First, we filter out all the renditions that have a higher bitrate
than the new measurement:

![Bitrate filtering](bitrate-switching-2.png)
![Bitrate filtering](images/bitrate-switching-2.png)

Then we get rid of any renditions that are bigger than the current
player dimensions:

![Resolution filtering](bitrate-switching-3.png)
![Resolution filtering](images/bitrate-switching-3.png)

We don't want to signficant quality drop just because your player is
one pixel too small, so we add back in the next highest
resolution. The highest bitrate rendition that remains is the one that
gets used:

![Final selection](bitrate-switching-4.png)
![Final selection](images/bitrate-switching-4.png)

If it turns out no rendition is acceptable based on the filtering
described above, the first encoding listed in the master playlist will
Expand Down
34 changes: 28 additions & 6 deletions docs/dash-playlist-loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,21 @@ The [DashPlaylistLoader][dpl] (DPL) is responsible for requesting MPDs, parsing

The [DPL] is written to be as similar as possible to the [PlaylistLoader][pl]. This means that majority of the public API for these two classes are the same, and so are the states they go through and events that they trigger.

### States

![DashPlaylistLoader States](images/dash-playlist-loader-states.nomnoml.svg)

- `HAVE_NOTHING` the state before the MPD is received and parsed.
- `HAVE_MASTER` the state before a media stream is setup but the MPD has been parsed.
- `HAVE_METADATA` the state after a media stream is setup.

### API

- `load()` this will either start or kick the loader during playback.
- `start()` this will start the [DPL] and request the MPD.
- `parseMasterXml()` this will parse the MPD manifest and return the result.
- `media()` this will return the currently active media stream or set a new active media stream.

### States

- `HAVE_NOTHING` the state before the MPD is received and parsed.
- `HAVE_MASTER` the state before a media stream is setup but the MPD has been parsed.
- `HAVE_METADATA` the state after a media stream is setup.

### Events

- `loadedplaylist` signals the setup of a master playlist, representing the DASH source as a whole, from the MPD; or a media playlist, representing a media stream.
Expand All @@ -40,6 +42,26 @@ The [DPL] is written to be as similar as possible to the [PlaylistLoader][pl]. T
- `mediachanging` signals that the currently active media stream is going to be changed.
- `mediachange` signals that the new media stream has been updated.

### Interaction with Other Modules

![DPL with MPC and MG](images/dash-playlist-loader-mpc-mg-sequence.plantuml.png)

### Special Features

There are a few features of [DPL] that are different from [PL] due to fundamental differences between HLS and DASH standards.

#### MinimumUpdatePeriod

This is a time period specified in the MPD after which the MPD should be re-requested and parsed. There could be any number of changes to the MPD between these update periods.

#### SyncClientServerClock

There is a UTCTiming node in the MPD that allows the client clock to be synced with a clock on the server. This may affect the results of parsing the MPD.

#### Requesting `sidx` Boxes

To be filled out.

### Previous Behavior

Until version 1.9.0 of [VHS], we thought that [DPL] could skip the `HAVE_NOTHING` and `HAVE_MASTER` states, as no other XHR requests are needed once the MPD has been downloaded and parsed. However, this is incorrect as there are some Presentations that signal the use of a "Segment Index box" or `sidx`. This `sidx` references specific byte ranges in a file that could contain media or potentially other `sidx` boxes.
Expand Down
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 12 additions & 0 deletions docs/images/dash-playlist-loader-states.nomnoml.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
26 changes: 26 additions & 0 deletions docs/images/playlist-loader-states.nomnoml.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
119 changes: 119 additions & 0 deletions docs/images/sources/dash-playlist-loader-mpc-mg-sequence.plantuml
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
@startuml

header DashPlaylistLoader sequences
title DashPlaylistLoader sequences: Master Manifest with Alternate Audio

Participant "MasterPlaylistController" as MPC #red
Participant "MasterDashPlaylistLoader" as MPL #blue
Participant "mainSegmentLoader" as SL #blue
Participant "AudioDashPlaylistLoader" as APL #green
Participant "audioSegmentLoader" as ASL #green
Participant "external server" as ext #brown
Participant "mpdParser" as parser #orange
Participant "mediaGroups" as MG #purple
Participant Tech #lightblue

== Initialization ==

MPC -> MPL : construct MasterPlaylistLoader
MPC -> MPL: load()

== Requesting Master Manifest ==

MPL -> MPL : start()

MPL -> ext: xhr request for master manifest
ext -> MPL : response with master manifest

MPL -> parser: parse manifest
parser -> MPL: object representing manifest

note over MPL #lightblue: trigger 'loadedplaylist'

== Requesting Video Manifest ==

note over MPL #lightblue: handling loadedplaylist
MPL -> MPL: media(x)

alt if no sidx
note over MPL #lightgray: zero delay to fake network request
else if sidx
break
MPL -> ext: request sidx
end
end

note over MPL #lightblue: trigger 'loadedmetadata' on master loader [T1]

note over MPL #lightblue: handling 'loadedmetadata'

opt vod and preload !== 'none'

MPL -> SL: playlist()
MPL -> SL: load()

end

== Initializing Media Groups, Choosing Active Tracks ==

MPL -> MG: setupMediaGroups()
MG -> MG: initialize()

== Initializing Alternate Audio Loader ==

MG -> APL: create child playlist loader for alt audio

MG -> MG: activeGroup and audio variant selected
MG -> MG: enable activeTrack, onTrackChanged()
MG -> ASL: reset audio segment loader

== Requesting Alternate Audio Manifest ==

MG -> MG: startLoaders()

MG -> APL: load()
APL -> APL: start()
APL -> APL: zero delay to fake network request

break finish pending tasks
MG -> Tech: add audioTrack
MPL -> MPC: setupSourceBuffers_()
MPL -> MPC: setupFirstPlay()

loop mainSegmentLoader.monitorBufferTick_()
SL -> ext: requests media segments
ext -> SL: response with media segment bytes
end
end

APL -> APL: zero delay over
APL -> APL: media(x)

alt if no sidx
note over APL #lightgray: zero delay to fake network request
else if sidx
break
MPL -> ext: request sidx
end
end

== Requesting Alternate Audio Segments ==

note over APL #lightblue: trigger 'loadedplaylist'
note over APL #lightblue: handling 'loadedplaylist'
APL -> ASL: playlist()


note over ASL #lightblue: trigger 'loadedmetadata' [T2]
note over APL #lightblue: handling 'loadedmetadata'
APL -> ASL: playlist()
APL -> ASL: load()

loop audioSegmentLoader.monitorBufferTick_()

ASL -> ext: requests media segments
ext -> ASL: response with media segment bytes

end

@enduml
21 changes: 21 additions & 0 deletions docs/images/sources/dash-playlist-loader-states.nomnoml.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#title: DASH Playlist Loader States
#arrowSize: 0.5
#bendSize: 1
#direction: down
#gutter: 10
#edgeMargin: 1
#edges: rounded
#fillArrows: false
#font: Arial
#fontSize: 10
#leading: 1
#lineWidth: 2
#padding: 20
#spacing: 50
#stroke: #33322E
#zoom: 1

#.label: align=center visual=none italic

[HAVE_NOTHING] load()-> [HAVE_MASTER]
[HAVE_MASTER] media()-> [HAVE_METADATA]
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loading

0 comments on commit 52201f9

Please sign in to comment.