Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: Add PDF support #88

Merged
merged 10 commits into from
Apr 11, 2024
Merged

feature: Add PDF support #88

merged 10 commits into from
Apr 11, 2024

Conversation

AhmadMuj
Copy link
Contributor

@AhmadMuj AhmadMuj commented Apr 8, 2024

  • Allowed PDF upload in both mobile and web
  • Dynamic type is being sent to the server based on the mimetype from both the mobile and web
  • Supported PDF view as iframe in both mobile and web
  • Added pdf-parse library to parse PDF files
  • Added content, info and metadata for the bookmarkAssets table to store the pdf parsing results so they can be indexed in the future. This would also be used in getting a description about the image and storing it in the content field.
  • Added inferTagsFromPDF that would parse the PDF, update the bookmarkAssets row and then inferTags from the text that includes the PDF content.

Copy link

vercel bot commented Apr 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hoarder-app-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 11, 2024 11:11am
hoarder-app-landing ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 11, 2024 11:11am

Copy link
Collaborator

@MohamedBassem MohamedBassem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow. I didn't see this coming!
First of all, thanks a lot for taking the time and contributing with this PR. I'm impressed with how you managed to navigate the different parts of the codebase without any help! As the first contributor to this project, please let me know if there's anything I can improve in the development documentation :)

The PR itself looks good to me, I requested some minor changes. Don't feel obliged to tackle the bigger ones, I'm happy to take over what you won't have time to fix!

I think the main missing thing is mobile rendering for the pdf type asset which lives in apps/mobile/components/bookmarks/BookmarkCard.tsx. I think for now, we can start with a static image saying that this is of type pdf and add a download button, later we can add the preview there.

apps/web/app/api/assets/route.ts Outdated Show resolved Hide resolved
apps/workers/openaiWorker.ts Outdated Show resolved Hide resolved
apps/workers/openaiWorker.ts Outdated Show resolved Hide resolved
apps/workers/openaiWorker.ts Outdated Show resolved Hide resolved
apps/workers/package.json Outdated Show resolved Hide resolved
apps/workers/searchWorker.ts Outdated Show resolved Hide resolved
packages/db/schema.ts Outdated Show resolved Hide resolved
apps/workers/openaiWorker.ts Outdated Show resolved Hide resolved
packages/db/schema.ts Show resolved Hide resolved
@AhmadMuj
Copy link
Contributor Author

AhmadMuj commented Apr 8, 2024

@MohamedBassem Kindly check now and let me know.
I'll add the images paste feature later after we close this.
I changed the prompt slightly to add a description for all content types. It's nice to have a summarization of the content. I didn't support indexing nor handling the content for now as I only worked on the prompt side for now.
The rest is handled

Copy link
Collaborator

@MohamedBassem MohamedBassem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I think this is good to go after addressing the prompt change comment! I'll take care of adding the mobile renderer in a separate PR.

It also seems that CI is failing, you'll need to run in the root of the repo:

pnpm typecheck && pnpm format:fix && pnpm lint:fix

to ensure that CI will pass when you push a new version.

EDIT: Also seems like the lockfile needs updating, you'll need to run pnpm i in the root as well.

apps/workers/openaiWorker.ts Outdated Show resolved Hide resolved
apps/workers/searchWorker.ts Outdated Show resolved Hide resolved
apps/workers/utils.ts Outdated Show resolved Hide resolved
apps/workers/utils.ts Outdated Show resolved Hide resolved
Copy link
Collaborator

@MohamedBassem MohamedBassem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR LGTM! We can land after fixing the asset schema and after you restore the preview fix that was lost in the merge commit!

packages/shared/assetdb.ts Outdated Show resolved Hide resolved
Copy link
Collaborator

@MohamedBassem MohamedBassem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is good to go! Thank you!

@MohamedBassem MohamedBassem merged commit be622e5 into hoarder-app:main Apr 11, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants