Add media segment skipping #6157

thornbill · 2024-10-02T21:10:55Z

Changes

Adds support for skipping media segments

Needs cleanup + testing so marking as a draft for now

Issues
N/A

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

github-actions · 2024-10-02T21:12:20Z

src/apps/stable/features/playback/utils/mediaSegmentManager.ts

+    private debouncedOnTimeUpdate = throttle(this.handleSegmentActions, 500, { leading: true });
+
+    private async fetchMediaSegments(api: Api, itemId: string, includeSegmentTypes: MediaSegmentType[]) {
+        // FIXME: Replace with SDK getMediaSegmentsApi function when available in stable


Unexpected 'fixme' comment: 'FIXME: Replace with SDK...'. no-warning-comments

This is dependent on the media segment api being available in the stable sdk release.

src/apps/stable/features/playback/utils/mediaSegmentManager.ts

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

viown · 2024-10-04T15:15:43Z

Just some things I noted when testing:

If I have "Skip" enabled and seek to a duration between the segment, it'll immediately take me to the end of the segment. If there is something there I wanted to take a look at I wouldn't be able to get to it. Maybe it should only skip when it hits the start time of the segment or perhaps avoid skipping altogether when the user manually seeks?
When it's about to skip a segment, it can look like the stream is frozen (this is amplified when transcoding). Having some kind of visual indicator (like the one for syncplay) would probably work well here.

solidsnake1298 · 2024-10-07T01:12:21Z

From my initial testing, this reliably skips intros, but it has not skipped outros or previews.

I'm going to create some test media so I can test the other segment types.

viown · 2024-10-07T10:30:42Z

From my initial testing, this reliably skips intros, but it has not skipped outros or previews.

I'm going to create some test media so I can test the other segment types.

It looks like all segments after the first will not be skipped because segment is not updated here:

jellyfin-web/src/apps/stable/features/playback/utils/mediaSegmentManager.ts

Line 47 in 91de060

    
           for (let index = lastIndex, segment = segments[index]; index >= 0 && index < segments.length; index += direction) {

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

solidsnake1298 · 2024-10-08T15:01:22Z

The latest commit is skipping intro, outro, and preview segments reliably. I haven't gotten a chance to create test media with other segment types, but I don't see why those wouldn't also skip.

solidsnake1298 · 2024-10-08T15:44:53Z

I created some test media for recap and commercial segments, which were also skipped.

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

thornbill · 2024-10-09T16:21:56Z

These are great points.

If I have "Skip" enabled and seek to a duration between the segment, it'll immediately take me to the end of the segment. If there is something there I wanted to take a look at I wouldn't be able to get to it. Maybe it should only skip when it hits the start time of the segment or perhaps avoid skipping altogether when the user manually seeks?

Good idea. I think this makes sense as an enhancement. We should align on the desired behavior with other client implementations (Android TV).

When it's about to skip a segment, it can look like the stream is frozen (this is amplified when transcoding). Having some kind of visual indicator (like the one for syncplay) would probably work well here.

I can definitely see this on my unstable server where I have assigned very limited resources. I wonder if just a toast message when skipping starts would be good enough for now. 🤔

nielsvanvelzen · 2024-10-09T16:34:29Z

If I have "Skip" enabled and seek to a duration between the segment, it'll immediately take me to the end of the segment. If there is something there I wanted to take a look at I wouldn't be able to get to it. Maybe it should only skip when it hits the start time of the segment or perhaps avoid skipping altogether when the user manually seeks?

Good idea. I think this makes sense as an enhancement. We should align on the desired behavior with other client implementations (Android TV).

This is how I defined the behavior for the skip action in ATV, from https://github.com/jellyfin/jellyfin-androidtv/blob/master/app/src/main/java/org/jellyfin/androidtv/ui/playback/segment/MediaSegmentAction.kt#L15-L16

	/**
	 * Seek to the end of this segment (endTicks). If the duration of this segment is shorter than 1 second it should do nothing to avoid
	 * lagg. The skip action will only execute when playing over the segment start, not when seeking into the segment block.
	 */
	SKIP,

	/**
	 * Ask the user if they want to skip this segment. When the user agrees this behaves like [SKIP]. Confirmation should only be asked for
	 * segments with a duration of at least 3 seconds to avoid UI flickering.
	 */
	ASK_TO_SKIP,

	/**
	 * Mute the audio volume until the end of this segment. The volume level should be restored to the state when this segment started. If
	 * the volume is manually changed by the user during this segment it should no longer restore volume itself. The mute action will always
	 * execute when the segment block is active.
	 */
	MUTE,

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

rlauuzo · 2024-10-12T08:43:30Z

I suggest adding a configurable buffer time (e.g., 2 seconds) at the start and end of segments to prevent accidental content skipping.

dmitrylyzo

Haven't tested - just reviewed. 😅

Am I right, it will skip the "outro" even we seek to it?

src/apps/stable/features/playback/utils/mediaSegments.ts

src/strings/en-us.json

Co-authored-by: Dmitry Lyzo <56478732+dmitrylyzo@users.noreply.github.com>

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

sonarcloud · 2024-10-13T07:44:03Z

Quality Gate passed

Issues
0 New issues
1 Accepted issue

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions

ESLint doesn't pass. Please fix all ESLint issues.

jellyfin-bot · 2024-10-13T07:47:12Z

Cloudflare Pages deployment

Latest commit	`aef9482824f22efd0e69c72f3ad8112657abd8cc`
Status	✅ Deployed!
Preview URL	https://cdb680bb.jellyfin-web.pages.dev
Type	🔀 Preview

thornbill · 2024-10-13T08:04:01Z

Am I right, it will skip the "outro" even we seek to it?

I just pushed a change that should bring the behavior inline with what Niels implemented for Android TV... I hope. 🤞

If you seek forward into a segment it will take the action as normal. If your previous player time is later than the start time of the segment no action is taken.

I made this behavior specific to the skip action because I think for others (prompting) it might still make sense to invoke the action. 🤷‍♂️

nielsvanvelzen · 2024-10-13T08:29:33Z

Skipping seems to work mostly as I'd expect except for one issue that I encountered. When I choose to skip outros and manually seek into an outro segment, it will be skipped. My expectation would be that manually seeking into it will not trigger the skip.

Otherwise it doesn't have some bugs that the current ATV implementation has, so that's great. (In ATV skipping to the end of the video will close the player instead of going to the next queue item :p)

thornbill · 2024-10-13T16:02:52Z

Yeah using the current time base approach we can only detect if we skip back into a segment not forward... I don't think we currently have any better way to detect when we have skipped unless we check if the difference between playtime updates is greater than some threshold. 🤷‍♂️

viown · 2024-10-13T16:30:06Z

Yeah using the current time base approach we can only detect if we skip back into a segment not forward... I don't think we currently have any better way to detect when we have skipped unless we check if the difference between playtime updates is greater than some threshold. 🤷‍♂️

Perhaps we could only trigger the skip if the current time is close to StartTicks by some difference (<1s), and also possibly checking lastTime < StartTicks?

dmitrylyzo · 2024-10-13T16:43:28Z

We will probably perform bigger steps with increased playback rate, and difference will depend on that.

The most reliable way is to add some extra flag to the timeupdate event, which is on when we set the current time of the player.

thornbill · 2024-10-13T17:05:13Z

I was thinking we could add an event on seeking in the playback manager and use that to ignore the segment if the next time update falls in one...

My question is if the current implementation is "good enough" for the initial PR or if this should be added before merging. 🤷‍♂️ I'm hoping to get this in for the next unstable build.

nielsvanvelzen · 2024-10-13T17:14:02Z

IMO current state is good enough to merge and we can improve upon it in 10.10.z releases.

needs stable sdk update

pimw1 · 2024-10-14T13:22:15Z

I've installed the "Chapter Segments" plugin and succesfully ran the "media segment scan" . For example, it was succesful for the below series episode.

[2024-10-14 15:02:52.631 +02:00] [INF] [24] Jellyfin.Server.Implementations.MediaSegments.MediaSegmentManager: Media Segment provider "Chapter Segments Provider" found 1 for "/data/media/tv/Criminal Minds/Season 03/Criminal Minds (2005) - S03E12 - 3rd Life [DSNP WEBDL-1080p][EAC3 5.1][h264]-playWEB.mkv"

When using https://cdb680bb.jellyfin-web.pages.dev/ to play this episode, i see the intro is recognized in the timeline

However, i don't get popup to skip the intro when the series progresses into that segment. How should this look like / behave?

dmitrylyzo · 2024-10-14T13:27:51Z

How should this look like / behave?

You need to select an action for the segment type in User settings / Playback. The action is applied without a prompt. In the future, there will be "Ask to skip".

pimw1 · 2024-10-14T13:37:42Z

Ah got it, thanks. It works as expected. I also notice that the transcoding is apparently taking it into account, since i don't notice the usual delay (which i would notice when manually skipping through the media). Nice!

pimw1 · 2024-10-14T13:39:36Z

Just a thought: perhaps introduce a global setting for skipping behaviour per segment type, which users can overwrite when they choose to. Reason: i don't see my non-techy family members enabling the skip behaviour.

pimw1 · 2024-10-14T14:30:20Z

Upon further testing, intro skipping sometimes does cause a tempory freeze due to transcoding. For a future update, it would be nice if transcoding would skip the intro anyway if the user setting is set to skip.

github-actions bot requested changes Oct 2, 2024

View reviewed changes

thornbill force-pushed the media-segment-actions branch from 9a4420b to 91de060 Compare October 2, 2024 21:13

github-actions bot requested changes Oct 2, 2024

View reviewed changes

thornbill force-pushed the media-segment-actions branch from 91de060 to 54713f9 Compare October 7, 2024 21:29

github-actions bot requested changes Oct 7, 2024

View reviewed changes

thornbill force-pushed the media-segment-actions branch from 54713f9 to f4c1684 Compare October 9, 2024 14:48

github-actions bot requested changes Oct 9, 2024

View reviewed changes

thornbill added this to the v10.10.0 milestone Oct 9, 2024

thornbill added the feature New feature or request label Oct 9, 2024

thornbill marked this pull request as ready for review October 9, 2024 16:14

thornbill requested a review from a team as a code owner October 9, 2024 16:14

jellyfin-bot added the merge conflict Conflicts prevent merging label Oct 9, 2024

This comment was marked as outdated.

Sign in to view

thornbill added 4 commits October 9, 2024 17:06

Add media segment action settings

809dba5

Add media segment manager

b934500

Add tests for segment utils

9aeb64e

Skip skipping if skip is short

7c4962d

thornbill force-pushed the media-segment-actions branch from cb9e1e4 to 7c4962d Compare October 9, 2024 21:07

jellyfin-bot removed the merge conflict Conflicts prevent merging label Oct 9, 2024

github-actions bot requested changes Oct 9, 2024

View reviewed changes

dmitrylyzo reviewed Oct 12, 2024

View reviewed changes

src/apps/stable/features/playback/utils/mediaSegments.ts Outdated Show resolved Hide resolved

src/apps/stable/features/playback/utils/mediaSegments.ts Outdated Show resolved Hide resolved

src/strings/en-us.json Outdated Show resolved Hide resolved

Apply suggestions from code review

9942fb7

Co-authored-by: Dmitry Lyzo <56478732+dmitrylyzo@users.noreply.github.com>

github-actions bot requested changes Oct 13, 2024

View reviewed changes

Prevent skipping when seeking back to segment

aef9482

github-actions bot previously requested changes Oct 13, 2024

View reviewed changes

nielsvanvelzen approved these changes Oct 13, 2024

View reviewed changes

thornbill merged commit a7185ed into jellyfin:master Oct 13, 2024
12 checks passed

thornbill deleted the media-segment-actions branch October 13, 2024 20:38

thornbill added the release-highlight A major new feature for the next release label Oct 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add media segment skipping #6157

Add media segment skipping #6157

thornbill commented Oct 2, 2024 •

edited

Loading

github-actions bot left a comment

github-actions bot Oct 2, 2024

thornbill Oct 9, 2024

github-actions bot left a comment

viown commented Oct 4, 2024

solidsnake1298 commented Oct 7, 2024

viown commented Oct 7, 2024

github-actions bot left a comment

solidsnake1298 commented Oct 8, 2024

solidsnake1298 commented Oct 8, 2024

github-actions bot left a comment

github-actions bot left a comment

thornbill commented Oct 9, 2024 •

edited

Loading

nielsvanvelzen commented Oct 9, 2024

This comment was marked as outdated.

github-actions bot left a comment

rlauuzo commented Oct 12, 2024

dmitrylyzo left a comment

github-actions bot left a comment

sonarcloud bot commented Oct 13, 2024

github-actions bot left a comment

jellyfin-bot commented Oct 13, 2024

thornbill commented Oct 13, 2024

nielsvanvelzen commented Oct 13, 2024

thornbill commented Oct 13, 2024

viown commented Oct 13, 2024 •

edited

Loading

dmitrylyzo commented Oct 13, 2024

thornbill commented Oct 13, 2024

nielsvanvelzen commented Oct 13, 2024

pimw1 commented Oct 14, 2024

dmitrylyzo commented Oct 14, 2024

pimw1 commented Oct 14, 2024

pimw1 commented Oct 14, 2024 •

edited

Loading

pimw1 commented Oct 14, 2024

Add media segment skipping #6157

Add media segment skipping #6157

Conversation

thornbill commented Oct 2, 2024 • edited Loading

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Oct 2, 2024

Choose a reason for hiding this comment

thornbill Oct 9, 2024

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

viown commented Oct 4, 2024

solidsnake1298 commented Oct 7, 2024

viown commented Oct 7, 2024

github-actions bot left a comment

Choose a reason for hiding this comment

solidsnake1298 commented Oct 8, 2024

solidsnake1298 commented Oct 8, 2024

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

thornbill commented Oct 9, 2024 • edited Loading

nielsvanvelzen commented Oct 9, 2024

This comment was marked as outdated.

github-actions bot left a comment

Choose a reason for hiding this comment

rlauuzo commented Oct 12, 2024

dmitrylyzo left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Oct 13, 2024

Quality Gate passed

github-actions bot left a comment

Choose a reason for hiding this comment

jellyfin-bot commented Oct 13, 2024

Cloudflare Pages deployment

thornbill commented Oct 13, 2024

nielsvanvelzen commented Oct 13, 2024

thornbill commented Oct 13, 2024

viown commented Oct 13, 2024 • edited Loading

dmitrylyzo commented Oct 13, 2024

thornbill commented Oct 13, 2024

nielsvanvelzen commented Oct 13, 2024

pimw1 commented Oct 14, 2024

dmitrylyzo commented Oct 14, 2024

pimw1 commented Oct 14, 2024

pimw1 commented Oct 14, 2024 • edited Loading

pimw1 commented Oct 14, 2024

thornbill commented Oct 2, 2024 •

edited

Loading

thornbill commented Oct 9, 2024 •

edited

Loading

viown commented Oct 13, 2024 •

edited

Loading

pimw1 commented Oct 14, 2024 •

edited

Loading