Video Block: Subtitles

[Soean]

Description

To add, edit and delete subtitles in the video block, we need an UI.

Current Classic Editor:

A video tag can have multiple tracks elements.
A track element has 4 Attributes.

• srclang (Valid BCP 47 language tag)
• label (Title for player UI)
• kind (Captions, Chapters, Descriptions, Metadata or Subtitles)
• src (URL of the text file)

Related: #7501

[karmatosed]

Can I ask how you get to this from the existing editor without a plugin? It would be great for steps so people can replicate and explore.

[Soean]

1. Add Video via 'Add Media'

2. Click on the edit button on top of the video
   

3. Add a Subtitle file

4. You see the Subtitle details
   

Here you can download example .mp4 and .srt files: https://github.com/mediaelement/mediaelement-files

[karmatosed]

This is a really rough mock but I am thinking something similar? That or using a pattern like playlist does could work?

[melchoyce]

An idea for adding subtitles:

The flag emoji would change based on the language detected in the subtitle file, since seems to be functionality we already have?

[Soean]

Hey @melchoyce
thanks for your design. I have a few comments on this.

• Flags are not ideal for this, because they represent states not languages, see http://www.flagsarenotlanguages.com/blog/why-flags-do-not-represent-language/
• There is no language detection, the default is english in the classic editor
• The user should be able to set these attributes for the track file:
  srclang (Valid BCP 47 language tag)
  label (Title for player UI)
  kind (Captions, Chapters, Descriptions, Metadata or Subtitles)
• I added the Accessibility label because it's new UI. Maybe we have to change the button labels? See Video Block: Update Poster Image UI #11991 (comment)

[Ipstenu]

Is there any movement on this at all?

As it stands I cannot add subtitles at all, let alone per language, to videos, which is not a good experience for my users :(

[paaljoachim]

What about something like this?

I just added a settings icon. Click the icon and it opens something to add the subtiles.

(I am working on keeping options outside of the sidebar.)

[afercia]

As commented on a similar issue, not sure the sidebar is the best place for tracks. Anything content related should be within the block.

And yes, flags are not languages 🙂Also, prefer plain text when possible instead of icons.

[anevins12]

@afercia In the most ideal scenario, where would you expect the tracks to appear? I think that's a really tough question to answer but should help drive this ticket forward

[afercia]

@anevins12 tough question. I don't even understand why clicking the "Edit" button (the pencil) switches back the editor to the "placeholder". Instead, in the Classic Editor it opens the details modal, where there's plenty of space for all the settings:

In Gutenberg the available space is way more limited. The sidebar is only 300 pixels wide. Also, we know it's very hard to use for persons with accessibility needs.

I'd like to see more settings in the "placeholder":

According to the Gutenberg design principles, the primary actions / settings should be in the block UI. I'd say alternate sources and tracks are directly related to content and as such are "primary".

[joedolson]

It would be great to be able to upload and assign subtitle/caption tracks directly in the placeholder. In addition to making an easier interface, it would also greatly increase people's awareness of the feature and improve the likelihood it gets used.

[melchoyce]

I'll try an updated version with the fields in the placeholder!

Any thoughts on whether the sidebar would feel like an appropriate place for changing or updating subtitles after the fact? My inclination still leans towards the sidebar as a good place to house subtitles, but I want to get y'alls a11y opinions first.

[joedolson]

You mean fields for switching out already-added subtitles for a different set of subtitles?

Sidebar might be OK as a place to change subtitles - while I think adding subtitles in the first place is a first-tier activity, I can see changing them as a secondary activity. It should be a considerably less frequent task.

That said, I do agree with @afercia that content changes should generally take place within the block itself - and captions & subtitles are definitely content. It seems like there should be ample space in the placeholder container or some other in-block interface to manage these.

[enriquesanchez]

I've created a first pass at a proposal and prototype. I've considered the following as requirements to meet:

1. A user should be able to add and remove subtitle tracks to a video
2. A video can have multiple tracks elements.
3. Each track element has 4 Attributes:

• srclang: (Valid BCP 47 language tag)
• label: (Title for player UI)
• kind: (Captions, Chapters, Descriptions, Metadata or Subtitles)
• src: (URL of the text file)

4. All of this, should be exposed from the block’s toolbar.

---

The following prototype covers the case of adding multiple tracks. Each track has the option to be edited or deleted.

The flow shown above would bring us to parity with what's currently available in the classic editor.

In order to make the editing of the track's attributes friendlier, we'd need to hide the track's markup and instead expose each attribute as a separate input field, as suggested above.

I'll continue exploring ideas around this.

For now, I'd appreciate feedback on the prototype I shared:

• Does it bring us to parity with the classic editor? Is there anything missing?
• How should we handle the case where there are a lot of tracks?
• Both in the classic editor and here, the prompt is to "Add subtitles", but we then call these "Tracks". Also, these tracks can have a different kind attribute: Captions, Chapters, Descriptions, Metadata, or Subtitles. Should we use a different term? Would "Tracks" be generally understood?

[paaljoachim]

Hey Enrique!

Does it bring us to parity with the classic editor? Is there anything missing?

An overview of todays method:

1. Add video.
   

2. The video is added to classic editor layout. Clicking edit brings up the Video Details screen. One then clicks the left sidebar - Add subtitles text link, and the title becomes Add Subtitles.

3. Subtitle is added to the video. Which is seen in the Video Details screen.

Based on the above Classic Editor. Now transferred to Gutenberg. After having added a video one then might want to add a subtitle.

It is really helpful to have a subtitle icon in the toolbar. Making it easy to upload/media library and edit inline the existing subtitle(s). Inline editing keeps it directly in place without having to go to another screen to edit the subtitle.

I did not see anything missing in relation to subtitles from your prototype.

How should we handle the case where there are a lot of tracks?

I am not sure what you mean with this (just send me a message and I can rephrase/edit this comment), but the way you have set it up with the subtitles icon with upload/media library works well for one or multiple tracks/subtitles.

Both in the classic editor and here, the prompt is to "Add subtitles", but we then call these "Tracks". Also, these tracks can have a different kind attribute: Captions, Chapters, Descriptions, Metadata, or Subtitles. Should we use a different term? Would "Tracks" be generally understood?

As I experience it tracks in relation to video can be confusing while subtitles is a common term.
For instance. Tracks on a CD. I would not say tracks in relation to a video. I did a search and it gives an idea how it is used: https://support.firecore.com/hc/en-us/articles/215090907-Subtitles-and-Audio-Tracks
Audio tracks and video subtitles. I do not think we would say audio subtitles and video tracks. As that is mixing the terms.

But as I read your comment again. It seems that video track is something above in the hierarchy of video subtitles. Tracks is used inside the inline editing of a subtitle and also used above the edit area. Having the word tracks so prominent just above the subtitles to me seems like a mistake. Instead of the word TRACKS it should say Subtitles, as right below the subtitles are linked. Inside the subtitle edit area it could for instance say attribute instead of tracks.

[joedolson]

"Tracks" is the most correct option here. The track element in HTML5 is used for subtitles and captions, but is also the element that provides support for chapters and other metadata. If this only said 'subtitles', it would strongly imply that it only supported that kind of track.

From a terminology standpoint, "tracks" should be considered more similar to tracks in audio mixing (e.g., parallel simultaneous tracks), rather than tracks on a CD (discrete sequential tracks.)

However, the UI certainly has room for improvement; it might be better to expose each type of track as an independent upload option matching the type of data. E.g., 'Subtitles', 'Captions', 'Chapters', etc.

However, that's not really necessary to reaching parity, and might be better as a later interface improvement.

[pablohoneyhoney]

Perhaps the bubble refers to chat or conversation, and we can have something more direct like a screen with two lines alluding to text. Like:

[enriquesanchez]

Thanks for the feedback @paaljoachim, @joedolson and @pablohoneyhoney! 🙏

I spent a little more time working on the UI and looking at ways to make the editing of track attributes a little friendlier. Here's a new prototype:

---

There's a new step after uploading a track file that lets the user modify the label, srclang and kind attributes:

This new UI also appears when the user chooses to 'Edit' a track.

There's a "(?)" in the "Source language" field label that I'm thinking can link to the W3C reference on language tags and how to use them.

The 'Kind' dropdown should let the user pick between Captions, Chapters, Descriptions, Metadata, or Subtitles.

---

• How does the prototype feel with this new step to edit track attributes?
• Are we improving the accessibility support compared to the Classic editor?

[StefanXhunga]

Hi All, thank you for your assistance!

[afercia]

Aside: Unsubscribing from this issue as, for what concerns accessibility, it's already in the good hands of @joedolson and @enriquesanchez.

[paaljoachim]

@enriquesanchez I really like the additional editing screen you made. Which makes it a lot easier to edit a track.
Thanks!

[enriquesanchez]

Here's an updated prototype. Same flow just some minor refinements to the UI.

---

I think this is in a good spot, it brings us to parity with the classic editor and makes editing of track attributes easier. Unless anyone thinks there are blockers, we should move forward with a PR and get it tested.

[joedolson]

This is a good starting point, and is an improvement on the equivalent process in the classic editor. I'd like the 'Edit' button to look more button like, but that's just design, and shouldn't stop this from moving forward.

[mtias]

Are "tracks" a clear enough term for users to use as the entry point? (Tooltip and heading within the dropdown menu.) I like the explanatory text, I think we should be using more of these contextually.

[enriquesanchez]

Are "tracks" a clear enough term for users to use as the entry point?

@mtias I'm not sure, as others mentioned above, while technically accurate the term feels more audio-related to me.

On the other hand, "subtitles" does not fully represent all the different types of tracks you can upload (captions, description, chapters, etc.).

We could call it "Captions & subtitles"? I've seen these two as the most common terms used for these text alternatives.

[enriquesanchez]

Looking at https://developer.mozilla.org/en-US/docs/Web/HTML/Element/track, maybe "Text tracks" is a better umbrella term for all of these types?

[enriquesanchez]

@Soean The design feels like in a good shape to start working on a PR. Would you have availability to work on this?

[joedolson]

I think that 'Text Tracks' seems like a good choice. It conveys more clearly the type of content that's in common to all of the supported fields without implying that it only supports a subset of those choices.

If it's called "Captions & subtitles", that would suggest that it doesn't support chapters or descriptions.

As far as I know, the player does not currently support descriptions, however.

[joedolson]

Would love to have a status update on this; this is a major accessibility gap in the feature set of Gutenberg, and if there's any chance it can get into a near-future release, I'd like to know that!

[jorgefilipecosta]

If no one is already working on an implementation of this feature, I can work on it :)

[enriquesanchez]

Thank you @jorgefilipecosta!

Please let me know if you have any question about the design and flow.

[jorgefilipecosta]

An implementation was proposed in #25861.

[enriquesanchez]

Would love to have a status update on this; this is a major accessibility gap in the feature set of Gutenberg, and if there's any chance it can get into a near-future release, I'd like to know that!

@joedolson The feature has just been merged 🎉

[joedolson]

Just saw that earlier today; happy to see it!