0.15.0

[MohamedBassem]

Welcome to the 0.15.0 release of Hoarder! It's been a while since the previous release, but this release is packed with a lot of new quality of life improvements and features. This releases wouldn't have been possible without the tons of contributions from @kamtschatka, @mdsaban & @AhmadMuj. I also want to welcome our first time contributors @kou029w & @devome. If you're interested in helping us develop hoarder, you can join our #development channel on hoarder's discord.

New Features

• Bulk actions are now a thing thanks to a contribution from @mdsaban. You can now bulk favourite, archive or delete bookmarks. In the next release, you'll be able to bulk add/remove bookmarks from lists.
• Full page archival is now here (powered by monolith). This will download everything that's in the page so that you can have an offline copy of that page even if that page disappears from the internet afterwards. The archives are typically quite large, so this feature is disabled by default. If you're a true hoarder, you can enable it with CRAWLER_FULL_PAGE_ARCHIVE=true.
• When you hoard a URL that points to a pdf or an image, Hoarder will now treat them as pdfs/images instead of links while keeping a reference to their original source. This will also allow automatic tagging to work correctly on those links. (thanks @kamtschatka!)
• You can now import your chrome bookmarks (or any Netscape compatible bookmark file) by exporting them from chrome, then drag and drop the exported file to hoarder. All links in the export will be imported. (thanks @kamtschatka!)
• For those hoarding code snippets, @kamtschatka added syntax highlighting to code blocks and a way to quickly copy the block.
• Hoarder is now available on arch linux's user repo (AUR) and is maintained by @devome. A new wiki page with installation instructions is now available.

UX Improvements

• @mdsaban revamped how both the user and admin settings pages look like.
• In the list layout, the editor will expand as you type longer notes thanks to @AhmadMuj.
• You can now paste images directly in the editor card and they'll get automatically uploaded thanks to @AhmadMuj.

Fixes

• The layout selector is now more clearly shown above the bookmark grid thanks to @mdsaban.
• Tags that were added manually were not correctly propagating to the search index. This is now fixed by @kamtschatka. You might want to trigger a reindex from the admin page if you've attached some manual tags.
• @kamtschatka fixed a long standing bug that prevented firefox users from using the browser extension without a valid HTTPS certificate.
• The editor card previously allowed you to save empty notes, this has now been fixed by @mdsaban.
• Increased list name max size to 40 chars up from 20.
• We previously had an upper limit on note sizes that we removed a while ago. However, we forgot to remove it when editing a note. @AhmadMuj fixed that.
• The CLI was previously printing invalid JSONs. Now there's a proper JSON mode thanks to @kamtschatka.
• For CJK environments, there's now a sans-serif fallback font thanks to @kou029w.
• Automatically inferred tags could have contained an extra space in the beginning of the tag, @kamtschatka fixed that.

Screenshots

Bulk Editing in Action

Screen_Recording_2024-06-26_at_10.46.49_AM.mp4

Revamped Admin Page

Revamped Settings Page

Full Page Archives

PDF/Image URL recognition

Screen.Recording.2024-07-06.at.11.50.14.AM.mov

Upgrading

To upgrade:

• If you're using HOARDER_VERSION=release, run docker compose pull && docker compose up -d.
• If you're pinning it to a specific version, upgrade the version and then run docker compose pull && docker compose up -d.

All Commits

• fix(web): Add sans-serif fallback for Inter font in CJK environments (fix(web): Add sans-serif fallback for Inter font in CJK character sets #272) by @kou029w in #72f03b2
• fix: Renaming full page archive asset file doesn't work inside docker. Fixes (Docker:nightly) Everything works but: "Crawling job failed: Error: EXDEV: cross-device link not permitted" leads to multiple retries and "failed crawling jobs" #273 by @MohamedBassem in #5e9b04a
• feat: Add bulk edit option for bookmarks. Fixes [Feature Request] Multi Select #84 ([Feat] Add bulk edit option for bookmarks #259) by @mdsaban in #d193d9b
• fix: fixed an issue that would half the textarea in grid layout after submitting a bookmark (fixed the editorcard textarea #269) by @kamtschatka in #bf92fa3
• refactor: fixed typos in BookmarkTypes (fixed typos #268) by @kamtschatka in #4e70fe3
• refactor: drag and drop improvements (Drag and drop improvements #264) by @kamtschatka in #9cd6170
• fix: Increase list name size to max 40 chars up from 20. Fixes New list name is limited to 20 characters #254 by @MohamedBassem in #9883c6b
• fix: Fix duplicate bookmarks in links with assets by @MohamedBassem in #3d495cd
• refactor: added the bookmark type to the database (Added the bookmark type to the database #256) by @kamtschatka in #e648646
• refactor: remove redundant code from crawler worker and refactor handling of asset types (refactoring asset types #253) by @kamtschatka in #ccbff18
• ui: refactor admin settings page ([UI/Refactor] Admin Settings #249) by @mdsaban in #e107f8b
• ui: Changes for user settings page ([UI] Changes for user settings page #251) by @mdsaban in #a637130
• feature: Automatically transfer image urls into bookmared assets. Fixes [Feature Request] Support for adding Images by URL #246 by @MohamedBassem in #1071095
• ui(web): move layout selector to main screen (Move layout control to main screen #252) by @mdsaban in #8f0e9b1
• refactor: extract assets into their own database table. Allow downloading more content from a webpage and index it #215 (Changes to have more assets #215 #220) by @kamtschatka in #9ce6958
• fix(landing): image width on landing page ([UI] Landing page Image breaking on safari #250) by @mdsaban in #0f54a18
• feature: add support for PDF links. Fixes feature request: pdf support #28 (feature request: pdf support #28 #216) by @kamtschatka in #be1b7f7
• fix(web): Fix save action on empty card. Fixes Empty bookmark #225 ([Bug] Fix save action on empty card #243) by @mdsaban in #ccfff6b
• ui: Improve landing page for homepage (Improve landing page for homepage #238) by @mdsaban in #16f21bf
• feature(web): Enhance the bookmark textarea in the list layout (feature: Enhance the bookmark textarea in the list layout #247) by @AhmadMuj in #ac05ecf
• readme: broken readme links (fix: broken readme links #233) by @mdsaban in #348bd80
• fix: Remove the bookmark update text limit (fix: Remove the bookmark update text limit #244) by @AhmadMuj in #671fc32
• fix(web): text paste bug introduced with image paste (fix: text paste bug introduced with image paste #232) by @mdsaban in #23faf46
• docs: add Arch Linux installation docs (add Arch Linux installation #223) by @devome in #dc19313
• feature(web): support pasting image in the Editor Card (feature: support pasting image in the textfield #228) by @AhmadMuj in #6823caf
• release(cli): Bump CLI version to 0.13.5 by @MohamedBassem in #219329b
• release(cli): Bump CLI version to 0.13.4 by @MohamedBassem in #5f263a3
• fix(cli): Bookmark list output is not a valid JSON. Fixes bookmark list output is not a valid JSON #150 (bookmark list output is not a valid JSON #150 #181) by @kamtschatka in #cde9726
• refactor: Extract the bookmark polling logic into a separate shared component by @MohamedBassem in #6928800
• fix: Trigger search re-index on bookmark tag manual updates. Fixes re-index of database is not scanning all places when bookmark tags are changed. Manual indexing is working as workaround #208 (re-index of database is not scanning all places when bookmark tags ar… #210) by @kamtschatka in #546139e
• feature(web): Add syntax highlighting to code blocks and a quick copy button. Fixes Any plans to support copy to clipboard (markdown code) for notes? #195 (Any plans to support copy to clipboard (markdown code) for notes? #195 #197) by @kamtschatka in #be1bb38
• fix(cli): Fix CLI version in help message. Fixes hoarder --version reports version 0.0.0 #139 by @MohamedBassem in #f7a7753
• feature(extension): Add a context menu to open your hoarder saves. Fixes Feature request: Browser extension "Open your Hoarder saves" #188 (Feature request: Browser extension "Open your Hoarder saves" #188 #206) by @kamtschatka in #2d0af0b
• fix(web): Changed the toast to contain a more detailed error message in case the bookmark is already part of the list. Fixes Web page shows generic error message if you try to add a duplicate bookmark #179 (more detailed error message in case the bookmark is already part of the list #179 #213) by @kamtschatka in #049e79d
• fix(workers): AI infered tags can contain " " at the beginning. Fixes AI infered tags can contain " " at the beginning #184 (AI infered tags can contain " " at the beginning #184 #194) by @kamtschatka in #be8b91f
• fix(crawler): Only update the database if full page archival is enabled by @MohamedBassem in #9d89f98
• docs: Document the new CRAWLER_FULL_PAGE_ARCHIVE flag by @MohamedBassem in #9198c1b
• feature: Full page archival with monolith. Fixes [Feature request] Store local copy of webpages #132 by @MohamedBassem in #dedc5fb
• feature: Allow import Netscape HTML format (Bookmark upload #163) by @kamtschatka in #033e8a2
• build: Let the upload-artificat action do the compression by @MohamedBassem in #bb431be
• release(extension): Bump extension version to 1.2.1 by @MohamedBassem in #8688dd4
• build: Add a workflow to build the extension by @MohamedBassem in #c9b1307
• fix(extension): Drop the https restriction in firefox. Fixes Firefox Plugin - NetworkError on Sign in #23 by @kamtschatka in #875cc35
• fix(cli): Switched to cursorV2 to make use of the fix for [Bug] Items created in the same second may not be shown in the bookmark grid due to pagination #140 and prevent infinite loops (Switched to cursorV2 to make use of the fix for #140 #176) by @kamtschatka in #6dc67cf

---

This discussion was created from the release 0.15.0.

[akshara-tg]

Thank you for another release with lot of enhancements.
My. .env file did not had "CRAWLER_FULL_PAGE_ARCHIVE=true" earlier. I have upgraded to the latest release and added "CRAWLER_FULL_PAGE_ARCHIVE=true" in my .env file.
I have around 1500 bookmarks. To prevent myself from 'link ROT' issues, which option should I use? "Recrawl All Links" Or "Reindex All Bookmarks" ?

Once again, thanks for all the hard work you guys do.

[kamtschatka]

Reindex will only update Meilisearch, what you want is a full recrawl.
Careful: I think that will also run tag inference again, so you might get additional new tags. You might want to disable that

[akshara-tg]

@kamtschatka Thanks for the swift response. Are you recommending I should use "Recrawl All Links (Without Inference)"? to avoid regenerating the tags using AI?
I curious to know whether "Recrawl All Links" will replace the existing tags with newly generated tags from AI inference OR will that append to the list of existing tags.

[kamtschatka]

ah, i did not remember that there is already such an option, Yes, you might want to use without inference then. Inference does not touch manually created tags, but the code seems to delete tags that did not come up again in the new inference, so it seems like it will replace existing tags with the newly infered ones.

[MohamedBassem]

@akshara-tg As @kamtschatka said, "recrawl without inference" is what you're looking for. If you're going to do a full page archive for 1500 links, you'll probably be looking at 10s of GBs of extra storage, so make sure you have enough space for that. I also recommend giving it a shot on a bunch of links first to check if that's what you're looking for before committing to doing it for 1500 links.

[akshara-tg]

@MohamedBassem Sure, I will try for few before recrawl the entire database. Appreciate that you took time reconfirm the approach. The option "Refresh" available for individual bookmark, would that re-fetch the entire webpage?

Thank you.

[undaunt]

Thanks for this, I've now fully replaced Shiori and Memos with the full archive feature.

[drgribb]

Thank you for Hoarder! It works fine and brings on a smile when I use it and realize that its controlled be me on MY server.

Its nice to see that its possible to manipulate entries in bulk.

A request: Is is possible to enable bulk edit when entries are recalled by tag?