Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: RAG Module #9

Merged
merged 34 commits into from
Jun 28, 2024
Merged

feat: RAG Module #9

merged 34 commits into from
Jun 28, 2024

Conversation

DaniAkash
Copy link
Contributor

@DaniAkash DaniAkash commented Apr 29, 2024

This pull request includes multiple changes to the codebase, primarily focused on enhancing the functionality of the App, Model, and Workflow classes, updating the package.json dependencies, and introducing a new constant in the rag.ts file.

The changes can be grouped into four main categories:

  1. Dependency updates: Two new dependencies were added to the package.json file. These include @types/tmp and llamaindex. [1] [2]

  2. App class enhancements: Several changes were made to the App class in the app.ts file. Notably, the AppConfig type was updated, and a new UserAppIDSet was imported and used within the class. The App class now includes a new info property, and several methods were updated to include a modelUserAppId parameter. [1] [2] [3] [4] [5] [6] [7] [8]

  3. Model class enhancements: The Model class in the model.ts file underwent several changes. The ModelConfig type was updated and a new UserAppIDSet was imported and used within the class. The Model class now includes a new modelUserAppId property, and several methods were updated to include a modelUserAppId parameter. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]

  4. Workflow class and other enhancements: The Workflow class in the workflow.ts file was updated to include new id and appId properties. A new constant MAX_UPLOAD_BATCH_SIZE was added to the rag.ts file, and the rag module was exported in the index.ts file. [1] [2] [3] [4]

Copy link
Contributor

github-actions bot commented Apr 30, 2024

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 64.77%
⬆️ +2.72%
4578 / 7068
🔵 Statements 64.77%
⬆️ +2.72%
4578 / 7068
🔵 Functions 67.91%
⬆️ +3.14%
127 / 187
🔵 Branches 63.82%
⬇️ -4.26%
344 / 539
File Coverage
File Stmts % Branch % Funcs % Lines Uncovered Lines
Changed Files
src/index.ts 100% 100% 100% 100%
src/client/app.ts 82.67% 62.26% 73.68% 82.67% 104-105, 108-113, 164, 209-210, 215-216, 220, 270-271, 273, 292-326, 343-377, 388-409, 412-413, 449-450, 491-492, 526-527, 552-555, 609-611, 660-661, 742-743, 770-771, 793-794, 816-817, 829-842, 862-863
src/client/model.ts 55.25% 46.51% 53.33% 55.25% 98-99, 101-102, 107-120, 129-130, 136-139, 147-181, 190-224, 235-303, 312-334, 343-399, 409-430, 441-444, 448, 465-466, 480-513, 540-543, 545-546, 556, 580, 582-585, 597-599, 631, 635, 704-705, 714-732, 774-778
src/client/rag.ts 80% 34.21% 100% 80% 62-65, 67-68, 72-75, 123-124, 127-128, 131-134, 140-141, 170-178, 189-194, 205, 223-224, 287-290, 297-300, 323-355, 397-400
src/client/workflow.ts 59.49% 47.36% 57.14% 59.49% 62-63, 65-66, 91-94, 108-111, 127, 129-132, 134-138, 144-146, 155-191, 194-213, 216-242, 270-273
src/client/auth/helper.ts 76.51% 72.09% 77.77% 76.51% 28-39, 42-44, 57-67, 160-176, 208-211, 222-223, 232-248, 258-259, 265-266, 290-291, 296, 299-302, 316-317, 353-356, 358-359
src/constants/rag.ts 100% 100% 100% 100%
src/rag/utils.ts 83.23% 42.3% 100% 83.23% 50-52, 56-57, 86-87, 94, 96, 100-102, 108-117, 130-131, 137, 142-144
src/urls/helper.ts 94.76% 90.47% 88.88% 94.76% 41-42, 113-116, 185-188
src/workflows/utils.ts 43.02% 80% 50% 43.02% 32-33, 39-58, 60-86
Generated in workflow #30

Base automatically changed from dani/datasets to main May 2, 2024 17:43
Copy link
Contributor

@alexandros-megas alexandros-megas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some of the type changes I mentioned would make this code much easier to understand as well as maintain.

I'd also like to see some examples for this stuff, because then I could look and see how I'm expected to consume these classes/methods as an end-user. This also makes it easier for us to see if we're creating something that's easy to use.

tests/client/rag.integration.test.ts Show resolved Hide resolved
src/client/model.ts Outdated Show resolved Hide resolved
src/client/rag.ts Show resolved Hide resolved
src/client/rag.ts Show resolved Hide resolved
@DaniAkash DaniAkash requested a review from brockdusome June 24, 2024 11:18
@DaniAkash
Copy link
Contributor Author

I think some of the type changes I mentioned would make this code much easier to understand as well as maintain.

I'd also like to see some examples for this stuff, because then I could look and see how I'm expected to consume these classes/methods as an end-user. This also makes it easier for us to see if we're creating something that's easy to use.

The examples have been published in https://docs.clarifai.com/sdk/rag

@brockdusome brockdusome dismissed alexandros-megas’s stale review June 28, 2024 13:46

Feedback was addressed, secondary approval acquired, removing to unblock merging.

@brockdusome brockdusome merged commit 7354e93 into main Jun 28, 2024
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants