MangaDex auto-updater?

[DuraMorte]

Would it be possible to keep the downloaded manga IDs in a table, and then scan the MangaDex site once a day to check for updates, and automatically download the updated chapters using the same criteria chosen originally? Maybe a checkbox for "Keep Updated" in the download manager?

[hkalexling]

This is indeed in the roadmap. Unfortunately, I had to relocate due to the coronavirus outbreak so I am struggling to find time to work on new features lately. I will keep this issue open and keep you updated.

[DuraMorte]

Fantastic! Stay safe and healthy, friend. :)

[flying-sausages]

I accidentally opened another issue as I didn't see this one while searching, so here are some thoughts I had in the other thread on how to getting these marked as "to download" right

Describe the solution you'd like
One of the following:

• Auto-download from the RSS of titles that are followed
• Mark shows as being followed and auto-download by scanning magadex for new chapters in an interval and retrieve the follwed titles either from:
  • Mangadex user through allowing access with credentials
  • Marking titles as followed in the interface manually

Additional context
I think the second option would be preferrable as you can then use the filter to specify groups, and/or make sure that only one copy is downloaded. You could also only keep the copy that is of the highest resolution.

[hkalexling]

I am returning to this issue, and I think the flow for this could be:

• Go to MangaDex and follow some manga
• Copy the RSS link (https://mangadex.org/rss/follows/long-rss-key) and paste it into the Mango subscription page
• Setup some filtering rules (e.g., manga name, chapter/volume range, groups, languages, dates, etc.)
• Mango would then periodically check the RSS and download matching chapters

The personal RSS link can be accessed without MangaDex credentials, so I think this is the easiest way. Any input is welcome!

[hkalexling]

Hmm, it looks like you can't do pagination with the RSS feed, so it only lists the latest 100 chapters. According to the devs on Discord, to get the older items you would have to use the API (https://api.mangadex.org/v2/user/me/followed-updates), which requires authentication :(

[flying-sausages]

which requires authentication :)

FTFY

[hkalexling]

@flying-sausages lol. The reason why I don't want authentication is that it doesn't make sense to have a shared MangaDex account for all users on a Mango server, so we can't just ask the admin to put their credentials in the config file. We will have to prompt each user to enter their username/password, and I can't think of a user-friendly way to do so. Storing credentials is another problem. MangaDex doesn't have API tokens so we will have to log users in with their username and password. I am not sure if we should just store the plain passwords in DB.

[flying-sausages]

You could probably fake a login, store the cookie and then save that. Present it as "connect to MD" and slap a big warning that cookies expire after 365 days

[hkalexling]

@flying-sausages Oh that's a great idea! I might just do that. Thanks!

[Baitinq]

If i might add to the login thing, i dont think the majority of users would mind storing an account name and password in plain text. This is an open source project so the code can be easily checked and it is an optional feature, so they could always just not use it :)

Mangadex at some point might add api tokens so this feature could just be advertised as a beta until they do so.

[fabiopbx]

In the case of auto-downloads, you can use the API without being logged in (except for x-rated stuff) if you know the id of the manga.. I made an entire vue app to do this and link back to mango, by making a request to get all chapters for that manga id and then work "some magic" to decide what's what and then queue up downloads of new chapters. So why not save the mangadex id and periodically do the search based on that.

I wish I knew crystal..

[hkalexling]

@fabiopbx It's less efficient that way. If you have 1000 manga downloaded from MangaDex, we would have to send 1000 requests to https://api.mangadex.org/v2/manga/{id}/chapters. With authentication, we can simply use https://api.mangadex.org/v2/user/me/followed-updates which returns every new chapter. We don't want to DDOS MD :)

For everyone following this, I have in fact already implemented the subscription/update features in the dev branch and planned to release it after more testings. But unfortunately with the recent hacking incident at MD, we would have to delay it. Hope they would bring the API back up soon.

[fabiopbx]

That does make sense, I never did look into what the v2/user/me/followed-updates offered much for the same reasons you mentioned, didn't want to store auth, ect...

But yh shame this whole hacking scene is happening :(

Been looking at the plugins to see if I can add some other sites...

I take it the auto-download is mangadex only then ?

[hkalexling]

@fabiopbx Yes, currently the advanced features (e.g., search manga by search terms and auto-update) are only available for MD. I do wish to move these features to plugins, so we can decouple from MD and just maintain it as a plugin. In this way we don't have to update the main app every time there's an API change, which it's likely to happen after they recover the site. But this requires some work and I need to find the time to do it.

[hkalexling]

@fabiopbx If you or anyone wanna help, we could start by brainstorming and designing the new version of plugin API (preferably in a new issue). The current version is documented at https://github.com/hkalexling/mango-plugins/wiki/Development-Guideline, and it only has the basic features. After we finalize the design I will implement it into the main app.

[fabiopbx]

@hkalexling Hello, I just managed to get the mangadex plugin working with very basic functionality, at least it can download, looking at the dev plugin docs are the v2 plugin methods already in Mango?

[hkalexling]

@fabiopbx Thanks for your effort! Unfortunately the v2 system is still WIP in the feature/plugin-v2 branch, and it will probably take a while to be completed :(

[tr7zw]

I'm gonna sneak in that if you feel like tinkering a bit, I have my Mango-Companion that I use to auto-download chapters from a decent selection of sites(also the updated mangadex), or plug them into mango with a plugin. But I'm not too sure about how complicated it is for others to set up, since so far (afaik) only I use it on my local nas. It also has some goodies like optionally sending a message into a Discord channel after it is done downloading new chapters.

[lucasmenno]

@tr7zw i would really like to use your creation, but my level of coding and building is not high enough..
Would you be so kind and tell me how i can get a .jar file. And/or maybe some other steps, please :)

[tr7zw]

@lucasmenno I can quickly look into it spitting out builds downloadable via github, give me a second.
Edit: it already does that, but currently not because a unit test is failing because some site changed its layout, so quickly need to fix that first.

[lucasmenno]

Alright. Thank you <3
damn, you are fast, big appreciation.

[tr7zw]

Email alerts and I was on Github at the same time looking for something else 😅

[tr7zw]

@lucasmenno Ok done, fixed one issue and turns out mangatx might be blocking the Github IPs, so the build was failing there. Now when clicking on the latest build there is an artifacts button (https://github.com/tr7zw/Mango-Companion/actions/runs/1518234563) to get the jar. Other than that, Java 8+ and the Github readme should be all required :D.

[lucasmenno]

Okay, got it working and downloading now. My server is blocking something (maybe the port, even when its 127.0.0.1) but my homeserver through zerotier got it working right away.
thank you for this quick troubleshooting and big thanks for the plugin :)

just need to figure out the way of automatically searching for new chapters.
i'm not a quick lerner :3

[tr7zw]

Bestcase contact me on the Mango Discord, then I can help better :D. But the tldr is to just add the URL of the manga to the config.json.

[fabiopbx]

For anyone who wants the plugin to download from the new mangadex api using the builtin download manager, I got it into a PR so you guys can have it :) :

getmango/mango-plugins#19
https://github.com/fabiopbx/mango-plugins/tree/feature/mangadexV5

[melyux]

Is this solved after v0.26.0? The Subscription Manager is there, though for MangaDex it always says "No subscriptions found." even though I've added subscriptions and can see them in plugins/mangadex/subscriptions.json.

[fabiopbx]

@melyux can confirm the auto download works, though it seems they don’t always show in the list, a refresh or so sometimes works. But they do auto update.

[hkalexling]

@melyux Yeah it should work. Make sure you are selecting the correct plugin in the dropdown. If it's still not working you can create a separate issue or discussion thread.

[hkalexling]

Closing this as the feature is now available.