Install publications (*.jwpub file)

[orangethewell]

Migration proposal

Last talked in commit 792d8b0, I have a migration proposal for database rust crate.

sqlite crate

This is a very simple crate that exposes a API for database handling using sqlite. The current version being used by OWL is 0.28.0, while the latest version is 0.36.1. The crate is usable, but updating it could be a hard task, since there's very little documentation available, and a lot of the code seems like a spaghetti mess, with SQLite statements everywhere and some neaty shortcuts.

Pros

• Simple, following KISS methodology;
• Straight to the point, get the data that you need, not a object;
• Every publication is PubCatalog responsability;

Cons

• Hard to maintain and keep data tracking;
• Lot of type casting inside the code;
• Publication data is messed up, since we get only the needed data.

microrm crate

Maybe we can say that microrm crate is quite advanced. It uses sqlx instead of being a standalone package, with this, it can be easier to maintain since there's isn't a lot of updates that change the code API. Is better documented, better maintained and it automatically refers data to object models.

Pros

• Easier to manage and access data;
• Straight to the point, get all things to use someday;
• Every publication is resposible for its own actions;
• PubCatalog turns into a database cluster instead of database caller;
• Can add and remove data from another type of databases;

Cons

• Since it isn't tested yet, can cause some damage to database models from publications;
• A lot of boilerplate to actually have something working well;
• Would require a publication API rewrite too;

Conclusion

Even that microrm requires more coding than sqlite, it seems like a very plausible reason to make the migration. It will help into making OWL code more organized and efficient. A development branch will be created for this.

[orangethewell]

So microrm crate is incompatible with publication tables, since publications are CamelCase formatted, and most of available ORMs there are async, let continue the search.

[orangethewell]

Since microrm wasn't compatible with publication database tables, now I will use rusqlite, which is a more recent binding to sqlite library and more feature rich than sqlite crate.

[orangethewell]

Last night I was trying to reverse engineer the Library App API and got some interesting finds, let me share with anyone reading this:

pub_collections.db

This seems to be a large database that holds publications installed (unordered, seems like something coming from the Publication table from every publications database, but maybe not), holds every MEPS reference of currently installed publications and their respective PublicationId for redirecting. There is some other stuff that I didn't have time to lookup.

/catalogs/media

This is an endpoint at the main API, every media, or every video data is held there. There are some steps to get it, since you have to get media languages available and then lookup with your language.

/mediator/*

I don't know where this is pointing at, I got this after disassembling the android app, and the others I got through a mitm proxy. It seems that the website also uses this, but I couldn't figure it out.

mepsunit.db

A single database with all data for MEPS language indexing

[orangethewell]

After some tests, everything about installing publications (for the long I know) is working fine. Old publications support also added, but I don't know if it will work nice on reading, but let see when finish feature/ui-development.