Support playlists with reasonable performance (proof of concept)

[anohren]

This loads a playlist of 1200+ items in 47 seconds on a Rpi 3. The videos can be played back sequentially, but does not support random access to playlist (queue) items using the Kodi GUI for some reason.

FIxes #15

[firsttris]

amazing. good work. like your idea.

you say
"but does not support random access to playlist items using the Kodi GUI for some reason"

does it play the next item after the first item is finished?

[anohren]

Yes, [1:] is the same string slicing syntax you used further up to remove the ? from the query string.

xbmc.Player() is supported in script addons. You have written a plug-in addon. The call initially caused a crash when the plugin recursed. I worked around that by making sure the first item in the queue was already resolved. Every other url is pointing back to the plugin itself. It hands itself YouTube-dl video ids ('urls') in this way.

Exactly. It works if you use the "next" button in the player.

[firsttris]

putting kodi in debug mode and checking log might help what is going wrong with random access?

[anohren]

That's an idea, didn't think of that . Although I think what's wrong is simply 'Kodi', since the plugin isn't even involved at that point.

It shouldn't be this hard. The documentation doesn't explain functions or types, and doesn't even list so called "properties" that you can set on list items. Not to mention explain their semantics. It's just a stab in the dark and an endless search on the forum.

That it's python doesn't exactly help 🙂 "a beautiful language" — if runtime exceptions are beautiful

[firsttris]

i feel like this is a super cool improvement but it also introduces a new bug. (random access of playlists)

i really want to test and look into this, but currently i have limited time

do you feel like this is ready for production or is it just a proof of concept ?

[anohren]

I would not release it.

It's not up to my standards as long as the bug is present. After all, what point is a long playlist if you can't jump in it?

I have yet to receive a reply in my thread about it in the Kodi forums.

[anohren]

Random access seems to work in Kodi Matrix on macOS. Just needs two quick fixes.

Not sure about what to do about earlier versions.

It requires a python 3 specific fix, and I'm not sure how/if you separate python2 and python3 versions somehow.

[anohren]

Should be python2/3 compatible now. I'll leave it to you @firsttris to evaluate backwards compatibility and requirements.

If the random access bug correlates with certain Kodi versions I suppose it would be possible to probe for that.

[firsttris]

will merge soon, just did some repository changes, where is splitted the repo for kodi 17 python 2 & 18 python3

[anohren]

Isn't it easier to be 2 & 3 compatible considering there are so few lines of code?

[firsttris]

i thought the source code is compatible with python 2 as well as 3, and its only a matter of the kodi xml files.

[anohren]

You know best about that. I just assumed you mentioned splitting up the repo in order to fork the code for some reason.

Does sound like more maintenance to have two repos though.

[firsttris]

the sourcecode remains in this repo, i meant there are now 2 different kodi addon repositories to install the extension from
https://github.com/firsttris/repository.sendtokodi.python3
https://github.com/firsttris/repository.sendtokodi

this was requested because the extension is now supported by the official android kodi app "kore".

[firsttris]

@nullket maybe if you find time you could also check & test this MR from @anohren which contains a POC to support big playlists with reasonable peformance.

let me know what you think, and if it makes sense to merge this to master.

[anohren]

@firsttris Do you mean to enable it only for Matrix and later? Random access play queue was broken as late as Leia. I have no idea why, but then I've found that the queue feels broken most of the time. It usually seems to just teleport me away somewhere at the slightest problem.

[anohren]

@firsttris
Did you manage to confirm the Kodi version cutoff point for erroneous play queue behavior with this PR?

[nullket]

@anohren I am picking up here to work on the playlists first.

I feel like I am overlooking something, can you help me here? Why is it faster with 'extract_flat': 'in_playlist' are the single playlist videos not resolved? But they must somewhere get resolved right...

If we start using urllib and distinguish between python2/3 we could even use the urlparse to get the playlist index for #60

We have a merge conflict and at least the sys import is obsolete. I wanted to create a new github action to build test .zip files on PR/commits anyway. Do you want to open a new PR as soon as we have those actions? That would make testing the addon a bit easier (what was the main problem why this stuck right?)

[anohren]

Why is it faster with 'extract_flat': 'in_playlist' are the single playlist videos not resolved? But they must somewhere get resolved right...

Yes, that's it.

This is how I explained it to Tristan:

xbmc.Player() is supported in script addons. You have written a plug-in addon. The call initially caused a crash when the plugin recursed. I worked around that by making sure the first item in the queue was already resolved. Every other url is pointing back to the plugin itself. It hands itself YouTube-dl video ids ('urls') in this way.

So the plug-in is called back for each item when it's actually played, to allow lazy evaluation.

If we start using urllib and distinguish between python2/3 we could even use the urlparse to get the playlist index for #60

Yeah I expect that the URL parsing will be more esthetic in Python than in Lua.

We have a merge conflict and at least the sys import is obsolete. I wanted to create a new github action to build test .zip files on PR/commits anyway. Do you want to open a new PR as soon as we have those actions? That would make testing the addon a bit easier (what was the main problem why this stuck right?)

Well, maybe convenience was the reason. But in all honesty it's just one file that you can paste right into while Kodi is running so I've never seen that as a big problem during development, and I've never imagined asking non-devs to help test — at least not such a simple thing as this.

I'll resolve this conflict if there's interest in moving the PR forward.

After that we would need a decision on:

• Whether there's a clear Kodi version above which the play queue always behaves properly, or some other obvious pattern.
• Whether we want to fix this problem conditionally for only those versions.

I think we should. Playlists are pretty pointless already if they take minutes to parse. As it currently stands, design-wise we're just conditioning users not to send playlists at all.

[anohren]

I think it's also interesting to keep in mind that SendToKodi should actually be a script addon instead of a plug-in addon:

Please do not confuse "Plugins" and "Scripts". It is very important to understand the difference between them before you start coding your addon. Unlike Scripts (which can basically perform any action), Plugins do not really provide new functionality to Kodi, instead what they do is provide a directory listing (for instance, like your movie library) to Kodi. This listing can contain folders as well as files. When a user clicks on a folder, kodi will run the plugin again to retrieve another listing. If a file is clicked, kodi will either play the file directly or call the addon to retrieve a playable url.
Another difference is that scripts can create their own gui (skin) while plugins can't. Plugin listings are presented in the current skin.
Do not try to mix and match script and plugin functionality as that will only give you weird error messages.

We're currently "mixing and matching".

I imagine that if we were a script instead we could do things like:

1. Receive a playlist URL
2. Start playing the first video as soon as possible (as already happens in this PR)
3. Continue to proactively resolve videos in the queue, so that there's no delay when playing the next item.

[nullket]

Sure lets work on this.

Thanks for the explanation. But just to be clear: So kodi calls the plugin with the url when one item in the playlist ends and the next entry wants to start? So it simply goes into the else (not a playlist) part and sets xbmcplugin.setResolvedUrl(__handle__, True, listitem=createListItemFromVideo(result))?

Whether there's a clear Kodi version above which the play queue always behaves properly, or some other obvious pattern.

From reading the above, what is the exact issue? What behaves wrong in which way? You have kodi 17 & 18 and I have kodi 19 we can get this tested, fixed and merged.

I think it's also interesting to keep in mind that SendToKodi should actually be a script addon instead of a plug-in addon:

Maybe we should open another issue for that and include a bit of refactoring etc.

I imagine that if we were a script instead we could do things like:

That sounds a lot cleaner to me. The downside would be that the youtube playlist with a starting index thing would get a bit more complicated again. We might not be able to add videos to a kodi playlist before the current playing one? Is this important at all? I mean it would be nice to have the whole youtube playlist but the "playlist so far" should be enough right?

[anohren]

Ok, I'll take a look at this then.

So it simply goes into the else

You've got it.

From reading the above, what is the exact issue? What behaves wrong in which way?

I think the best description I can give is "the play queue implodes".

I haven't tried it for some time, but when I select a queued item the entire queue in the GUI is replaced with some broken version of your standard Kodi library/file browser. Or it all just disappears -- I don't remember exactly which. You'll know it when you see it.

Maybe we should open another issue for that and include a bit of refactoring etc.

Sure. It can't hurt to play around with the idea at least. I don't know exactly what it would entail since I've never written a Kodi addon script.

That sounds a lot cleaner to me. The downside would be that the youtube playlist with a starting index thing would get a bit more complicated again. We might not be able to add videos to a kodi playlist before the current playing one? Is this important at all? I mean it would be nice to have the whole youtube playlist but the "playlist so far" should be enough right?

I just made sure that queued items before the currently playing one are at least retained in the queue. That should be enough for our purposes regardless of if we're a script or a plugin. Preloading is just best effort, and the default will always be to resolve URLs lazily.

It's a bit sad how neglected (personal opinion) the Kodi play queue is -- it feels like it's been hidden away under an obscure button in an invisible side bar far outside of the actual video player screen ever since it ran on the original Xbox. If it were simly exposed on the player screen, the same way the Plex addon does it, it would be about 100 times more useful.

[nullket]

Ok, I'll take a look at this then.

Perfect! I just integrated the building as I am far too lazy to copy code around for testing!

purposes regardless of if we're a script or a plugin

Are you sure? In my understanding a script runs just one time e.g. when we submit the url initially and we would then have to parse all URLs in the background and not in a lazy way on demand? A plugin surely enabled lazy resolving as discussed above.

It's a bit sad how neglected (personal opinion) the Kodi play queue is

I get what you mean but I do not use them at all. I don't know why, maybe because they are the way the are? Maybe it is just not my style. Anyway I am more than happy to resolve the issues anyway maybe this brings enough motivation to work on input stream adaptive..

[anohren]

What the... Now random access works in 18.9 Media Center Kodi on OSMC.

Kodi 17.6 couldn't run it due to Error Contents: 'xbmcgui.ListItem' object has no attribute 'setIsFolder'. Huh. The method was added in 18. Removing that usage (added in this PR) doesn't seem to affect behavior anyway. And after that random access play queue works in Kodi for Debian 17.6 as well.

Well, I can't seem to recreate the problem anymore on OSMC (Rpi3) or 32 bit Ubuntu 18.04.

[anohren]

Last problem I experienced while testing is that custom youtube-dl options that were passed with the initial call to the plugin are not preserved when jumping to the other playlist entries. That's to be expected. They're not passed along with the playlist items.

I guess they can just be added to the path field of the ListItem in the same way the initial call does it (i.e. separated by whitespace).

Update: Fixed. This should be ready for testing now.

[nullket]

I have testes the build artifact of the latest commit on kodi 19.

• The playlists loads fine and starts with the first video
• I can open the playlist menu and browse the other videos
• As soon as I select the video something crashes and I am stuck in an infinite loop:

It seems like the url is percent encoded when submitted to the plugin.

Expand to see kodi log

2022-05-24 07:56:20.250 T:21334 WARNING <general>: Skipped 1 duplicate messages..
2022-05-24 07:56:20.250 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:27.555 T:21753    INFO <general>: Skipped 140 duplicate messages..
2022-05-24 07:56:27.555 T:21753    INFO <general>: initializing python engine.
2022-05-24 07:56:27.569 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:27.605 T:21335    INFO <general>: CVideoPlayerAudio::Process - stream stalled
2022-05-24 07:56:27.619 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:27.630 T:21755    INFO <general>: COutput::OnStartup: Output Thread created
2022-05-24 07:56:27.645 T:21334    INFO <general>: VAAPI::Close
2022-05-24 07:56:27.645 T:21755    INFO <general>: COutput::OnExit: Output Thread terminated
2022-05-24 07:56:27.649 T:21756    INFO <general>: COutput::OnStartup: Output Thread created
2022-05-24 07:56:27.808 T:1081     INFO <general>: VAAPI::Close
2022-05-24 07:56:27.808 T:21340    INFO <general>: COutput::OnExit: Output Thread terminated
2022-05-24 07:56:28.327 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:30.533 T:21753   ERROR <general>: Skipped 38 duplicate messages..
2022-05-24 07:56:30.533 T:21753   ERROR <general>: WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
                                                   
2022-05-24 07:56:30.552 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:32.190 T:21753   ERROR <general>: Skipped 32 duplicate messages..
2022-05-24 07:56:32.190 T:21753   ERROR <general>: ERROR: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M" ) to search YouTube
                                                   
2022-05-24 07:56:32.192 T:21753   ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'lib.yt_dlp.utils.DownloadError'>
                                                   Error Contents: ERROR: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M" ) to search YouTube
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/extractor/common.py", line 642, in extract
                                                       ie_result = self._real_extract(url)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/extractor/generic.py", line 2698, in _real_extract
                                                       raise ExtractorError(
                                                   lib.yt_dlp.utils.ExtractorError: 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M" ) to search YouTube
                                                   
                                                   During handling of the above exception, another exception occurred:
                                                   
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1399, in wrapper
                                                       return func(self, *args, **kwargs)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1469, in __extract_info
                                                       ie_result = ie.extract(url)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/extractor/common.py", line 668, in extract
                                                       raise type(e)(e.orig_msg, **kwargs)
                                                   lib.yt_dlp.utils.ExtractorError: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M" ) to search YouTube
                                                   
                                                   During handling of the above exception, another exception occurred:
                                                   
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/service.py", line 150, in <module>
                                                       result = ydl.extract_info(url, download=False)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1390, in extract_info
                                                       return self.__extract_info(url, self.get_info_extractor(ie_key), download, extra_info, process)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1417, in wrapper
                                                       self.report_error(str(e), e.format_traceback())
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 939, in report_error
                                                       self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 882, in trouble
                                                       raise DownloadError(message, exc_info)
                                                   lib.yt_dlp.utils.DownloadError: ERROR: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M" ) to search YouTube
                                                   -->End of Python script error report<--
                                                   
2022-05-24 07:56:32.241 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:32.484 T:21753    INFO <general>: Skipped 4 duplicate messages..
2022-05-24 07:56:32.484 T:21753    INFO <general>: Python interpreter stopped
2022-05-24 07:56:32.497 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:32.517 T:1081    ERROR <general>: Playlist Player: skipping unplayable item: 4, path [plugin://plugin.video.sendtokodi/?https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D406HfBHm-4M  {"ydlOpts":{}}]
2022-05-24 07:56:32.518 T:21897    INFO <general>: initializing python engine.
2022-05-24 07:56:32.548 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:35.745 T:21897   ERROR <general>: Skipped 61 duplicate messages..
2022-05-24 07:56:35.745 T:21897   ERROR <general>: WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
                                                   
2022-05-24 07:56:35.763 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:35.779 T:21335    INFO <general>: CVideoPlayerAudio::Process - stream stalled
2022-05-24 07:56:35.814 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:35.818 T:22055    INFO <general>: COutput::OnStartup: Output Thread created
2022-05-24 07:56:35.831 T:21334    INFO <general>: VAAPI::Close
2022-05-24 07:56:35.831 T:22055    INFO <general>: COutput::OnExit: Output Thread terminated
2022-05-24 07:56:35.835 T:22056    INFO <general>: COutput::OnStartup: Output Thread created
2022-05-24 07:56:35.853 T:1081     INFO <general>: VAAPI::Close
2022-05-24 07:56:35.853 T:21756    INFO <general>: COutput::OnExit: Output Thread terminated
2022-05-24 07:56:36.373 T:21334 WARNING <general>: OutputPicture - timeout waiting for buffer
2022-05-24 07:56:37.565 T:21897   ERROR <general>: Skipped 19 duplicate messages..
2022-05-24 07:56:37.565 T:21897   ERROR <general>: ERROR: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM" ) to search YouTube
                                                   
2022-05-24 07:56:37.567 T:21897   ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'lib.yt_dlp.utils.DownloadError'>
                                                   Error Contents: ERROR: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM" ) to search YouTube
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/extractor/common.py", line 642, in extract
                                                       ie_result = self._real_extract(url)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/extractor/generic.py", line 2698, in _real_extract
                                                       raise ExtractorError(
                                                   lib.yt_dlp.utils.ExtractorError: 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM" ) to search YouTube
                                                   
                                                   During handling of the above exception, another exception occurred:
                                                   
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1399, in wrapper
                                                       return func(self, *args, **kwargs)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1469, in __extract_info
                                                       ie_result = ie.extract(url)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/extractor/common.py", line 668, in extract
                                                       raise type(e)(e.orig_msg, **kwargs)
                                                   lib.yt_dlp.utils.ExtractorError: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM" ) to search YouTube
                                                   
                                                   During handling of the above exception, another exception occurred:
                                                   
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/service.py", line 150, in <module>
                                                       result = ydl.extract_info(url, download=False)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1390, in extract_info
                                                       return self.__extract_info(url, self.get_info_extractor(ie_key), download, extra_info, process)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 1417, in wrapper
                                                       self.report_error(str(e), e.format_traceback())
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 939, in report_error
                                                       self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/lib/yt_dlp/YoutubeDL.py", line 882, in trouble
                                                       raise DownloadError(message, exc_info)
                                                   lib.yt_dlp.utils.DownloadError: ERROR: [generic] 'https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYWiwVaP0JQM" ) to search YouTube
                                                   -->End of Python script error report<--

[anohren]

I don't think I'm even going to think about this further than WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this. which makes me believe it's Kodi interfering with everything. Again.

Let's try that env variable first of all and see what happens. Then if it works we'll figure out why it thinks out URL is a file system path.

Update: seems this message is logged by YouTube-dl. Are we sure that not YouTube-dl is returning these mangled URLs? It's kind of strange since we're using the result's url field and not some field like filename...

Maybe we'll try using the video ID field instead and see where that takes us.

@nullket Oh and what environment are you running in, which exact Kodi version, and which playlist?

[nullket]

which makes me believe it's Kodi interfering with everything. Again
seems this message is logged by YouTube-dl.

Yes I also have the feeling that it might be kodis fault but the LC_ALL warning is coming from yt_dlp.

Are we sure that not YouTube-dl is returning these mangled URLs?

I did a quick test in ipython, it seems to work correct.

In [5]: url = "https://youtube.com/playlist?list=PLr_CZLgMkHeUexzcA64PfjCW0UUoO8Twi" 
In [6]: dl_opts = {'format': 'best', 'extract_flat': 'in_playlist'}  
In [7]: result = ydl.extract_info(url, download=False)                                                                                      
In [8]: result['entries'][2]['url']                                                                                                                                          
Out[8]: 'https://www.youtube.com/watch?v=KzSoabcQYMg'
In [9]: result['entries'][2]['url']                                                                                                                                          
Out[10]: 'https://www.youtube.com/watch?v=YWiwVaP0JQM'

which playlist?

https://youtube.com/playlist?list=PLr_CZLgMkHeUexzcA64PfjCW0UUoO8Twi

environment are you running

• 19.4 (19.4.0) Git:20220304-e12e66e019 Media Center Kodi
• Ubuntu 20.04 Server
• Started with systemd by a kodi user, here is the unit file

[Unit]
Description=Starts instance of Kodi using xinit
After=systemd-user-sessions.service network-online.target sound.target
Requires=network-online.target
Conflicts=getty@tty1.service

[Service]
User=kodi
Group=kodi
PAMName=login
TTYPath=/dev/tty1
Environment="LC_ALL=C.UTF-8"
ExecStart=/usr/bin/xinit /usr/bin/kodi-standalone -- :0 -nolisten tcp vt1
Restart=on-abort
StandardInput=tty

[Install]
WantedBy=multi-user.target

[anohren]

Nice, thanks. I'll see if I can recreate it.

[anohren]

Hmm. I tried but failed, and when inspecting it I was apparently not testing this PR. Did I misunderstand where to install from? I just pulled 0.9.171 from the python3 repo.

[anohren]

Okay I've tried it, and could reproduce the issue.

It only happens when using yt_dlp, so I'll assume that your manual test was mistakenly with YouTube-dl, which was my suspicion.

I'll preliminarily conclude that yt_dlp does sanity checks that are reaching outside its scope. If this is the case then the assumption that output in a field called url will be used as a file name is just wrong.

Update: and apparently my suspicion was unfounded — I can't recreate it manually with yt_dlp.

[nullket]

Did I misunderstand where to install from? I just pulled 0.9.171 from the python3 repo.

Yes, I am not releasing an unmerged PR for everybody. Follow the checks here in the PR or go to the actions tab and choose the latest run for your PR/branch. Download the correct zip (for python3 it would be "Addon-artifacts-Matrix", extract the download once to get the zipped plugin (there is no way to avoid the double zipping caused by github actions)

https://github.com/firsttris/plugin.video.sendtokodi/actions/runs/2373042603

It only happens when using yt_dlp, so I'll assume that your manual test was mistakenly with YouTube-dl, which was my suspicion.

Sorry if I was unclear about that I aborted testing after it did not work with yt_dlp.

Update: and apparently my suspicion was unfounded — I can't recreate it manually with yt_dlp.

That is why I also tested it real quick with a standalone python session.

So just to clarify (I am not at home to test), does it work with a kodi matrix instance (I assume you now have one for testing?) and youtube_dl?

[anohren]

So just to clarify (I am not at home to test), does it work with a kodi matrix instance (I assume you now have one for testing?) and youtube_dl?

Yes, exactly, weirdly it works when youtube-dl is selected in the settings. That's what's throwing me off.

It's easy to fix, but I'm still a bit weary about putting arbitrarily formatted strings into path fields of Kodi components and expecting to have them returned to me untouched.

[nullket]

It's easy to fix, but I'm still a bit weary about putting arbitrarily formatted strings into path fields of Kodi components and expecting to have them returned to me untouched.

Yes, me too but we still do not really know if it is not yt_dlp (I mean it works outside of kodi but who knows...). Can we look at the listItem after we created it and simply print it to the log for debugging to see if it is already dangled when we put it in the list?

[anohren]

If this works for everyone I think we're ready to merge. Let's test random access in the play queue once more to make sure nothing else has been overlooked:

• Kodi 17, 18, 19
• A Youtube playlist (https://youtube.com/playlist?list=PLr_CZLgMkHeUexzcA64PfjCW0UUoO8Twi) and a non-Youtube playlist (i.e. a webpage embedding multiple videos, e.g. https://benheck.com)
• Using youtube-dl and yt-dlp

[nullket]

Let's test random access in the play queue once more
Youtube seems to work fine now but your suggested website does not. It seems like the generic scrapper does not provide a title key?

2022-05-27 12:03:32.669 T:14862    INFO <general>: initializing python engine.
2022-05-27 12:03:35.463 T:14862   ERROR <general>: WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
                                                   
2022-05-27 12:03:37.366 T:14862   ERROR <general>: WARNING: [generic] Falling back on generic information extractor.
                                                   
2022-05-27 12:03:39.038 T:14862   ERROR <general>: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                                    - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                                   Error Type: <class 'KeyError'>
                                                   Error Contents: 'title'
                                                   Traceback (most recent call last):
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/service.py", line 157, in <module>
                                                       list_item = createListItemFromFlatPlaylistItem(video)
                                                     File "/home/kodi/.kodi/addons/plugin.video.sendtokodi/service.py", line 90, in createListItemFromFlatPlaylistItem
                                                       title = video['title']
                                                   KeyError: 'title'
                                                   -->End of Python script error report<--
                                                   
2022-05-27 12:03:39.445 T:14862    INFO <general>: Python interpreter stopped

[anohren]

Youtube seems to work fine now but your suggested website does not. It seems like the generic scrapper does not provide a title key?

Good catch. Fixed. Weird that it doesn't work now — it seemed to extract titles just fine when I tested these changes in the addon installed directly from the repo. Maybe I overlooked something.

[nullket]

@anohren thanks for your work and especially your precision on the subject! It worked for me, I assume it worked for you so I merged it and now we will see if anybody (at all) has a problem ;)

[anohren]

Thank you for helping to move it forward and for trying it out! Sounds like a good plan.