Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add mocked test & benchmark for LLM agents pipeline #1424

Merged
merged 44 commits into from
Dec 14, 2023
Merged

Add mocked test & benchmark for LLM agents pipeline #1424

merged 44 commits into from
Dec 14, 2023

Conversation

dagardner-nv
Copy link
Contributor

@dagardner-nv dagardner-nv commented Dec 12, 2023 •
edited
Loading

Description

  • Add a test for the LLM agents pipeline that doesn't communicate with OpenAI or SerpAPI.
  • Add benchmark for LLM agents pipeline.

Includes changes from PR #1421

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@dagardner-nv
List of UN member nations based upon https://www.un.org/dgacm/sites/w…
4eaac3f 
…ww.un.org.dgacm/files/Documents_Protocol/officialnamesofcountries.pdf
@dagardner-nv
Add heading row
0c43db8
@dagardner-nv
fix heading row case
357e657
@dagardner-nv
Add flags to read in the input from a file, and optionally shuffle th…
b5f4b4a 
…e dataframe
@dagardner-nv
Move nemo and openai fixtures up a level to be shared with benchmarks
9e8d4ca
@dagardner-nv
Fix integration test, responses are prefixed with text
575c805
@dagardner-nv
Extend existing mock_chat_completion and mock_nemollm fixtures to ran…
a8876db 
…domly sleep
@dagardner-nv
Call asyncio.get_event_loop() rather than using the event_loop fixtur…
8e0f99c 
…e, avoids issue where the futures were created on a different loop than the llm engine
@dagardner-nv
Add missing --run_benchmark flag
e58aeb5
@dagardner-nv
Add benchmark for the completion pipeline
2918bd6
@dagardner-nv
Rename benchmark to avoid name clash
477ca6d
@dagardner-nv
Remove unused importd
a03e659
@dagardner-nv
Specify string as a regex string
fe12b9c
@dagardner-nv
Rename benchmark to match existing names
e58cd92
@dagardner-nv
Rename test
ed212ae
@dagardner-nv
WIP: update docker run
9a60b14
@dagardner-nv
WIP: benchmark for vdb upload pipeline
4a0ec7e
@dagardner-nv
WIP
f97432d
@dagardner-nv
WIP
6e01776
@dagardner-nv
Cleanups
5771492
@dagardner-nv
Update docs
2325315
@dagardner-nv
Replace usage of random numbers with a fixed sleep based on average o…
4ee5e47 
…bserved sleep times
@dagardner-nv
Remove unused import
5acbb2e
@dagardner-nv
Merge branch 'david-24.03-llm-bench' into david-24.03-llm-bench-vdb_u…
bc675da 
…pload
@dagardner-nv
Merge branch 'branch-24.03' of github.com:nv-morpheus/Morpheus into d…
64e52a0 
…avid-24.03-llm-bench-vdb_upload
@dagardner-nv
Move populate_milvus helper method to be shared
9a9d5c7
@dagardner-nv
Fix LLMService type-hints
49acded
@dagardner-nv
Benchmark for stand-alone rag pipeline
e072be9
@dagardner-nv
WIP
6093394
@dagardner-nv
Move the impl for the import_mod fixture to it's own method allowing …
7c58396 
…it to be re-used
@dagardner-nv dagardner-nv requested a review from a team as a code owner December 12, 2023 23:42
@dagardner-nv dagardner-nv marked this pull request as draft December 12, 2023 23:43
@dagardner-nv dagardner-nv self-assigned this Dec 12, 2023
@dagardner-nv dagardner-nv added non-breaking Non-breaking change improvement Improvement to existing functionality Merge After Dependencies PR is completed and reviewed but depends on another PR; do not merge out of order labels Dec 12, 2023
@dagardner-nv dagardner-nv changed the title Draft: Adding mocked test & benchmark for LLM agents pipeline Add mocked test & benchmark for LLM agents pipeline Dec 13, 2023
@dagardner-nv dagardner-nv marked this pull request as ready for review December 13, 2023 00:05
@dagardner-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit fbc68de into nv-morpheus:branch-24.03 Dec 14, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mdemoret-nv mdemoret-nv mdemoret-nv approved these changes

Assignees

@dagardner-nv dagardner-nv

Labels
improvement Improvement to existing functionality Merge After Dependencies PR is completed and reviewed but depends on another PR; do not merge out of order non-breaking Non-breaking change
Projects
Morpheus Boards
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dagardner-nv @mdemoret-nv