Add Groq Llama3 support

[randy-tsukemen]

User description

• Adds support for GROQ.KEY in the LiteLLMAIHandler.
• Includes instructions in the usage guide for configuring the Groq Llama3 model.
• Adds the Groq API key configuration to the .secrets_template.toml file.

---

Type

enhancement

---

Description

• Added support for Groq Llama3 models in the system, including API key integration and model configurations.
• Updated the usage guide with detailed instructions on configuring the Groq Llama3 model.
• Included a new section in the .secrets_template.toml for Groq API key configuration.

---

Changes walkthrough

	Relevant files
Enhancement	__init__.py Add Groq Llama3 model configurations                                          pr_agent/algo/init.py Added Groq Llama3 model configurations with specific model keys. +2/-0      litellm_ai_handler.py Integrate GROQ.KEY support in LiteLLMAIHandler                      pr_agent/algo/ai_handlers/litellm_ai_handler.py Integrated GROQ.KEY support in LiteLLMAIHandler for API key management. +2/-0
Documentation	additional_configurations.md Update usage guide with Groq Llama3 model configuration    docs/docs/usage-guide/additional_configurations.md Added instructions for configuring Groq Llama3 model in the usage guide. +13/-0
Configuration changes	.secrets_template.toml Add Groq API key configuration to secrets template              pr_agent/settings/.secrets_template.toml Added Groq API key configuration section. +3/-0

---

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Description updated to latest commit (d457fa2)

• Copy walkthrough table to "Files Changed" Tab

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Review

⏱️ Estimated effort to review [1-5]	2, because the changes are straightforward and localized to specific files, involving configuration updates and minor code additions.
🏅 Score	85
🧪 Relevant tests	No
🔍 Possible issues	Duplicate Code: The line litellm.replicate_key = get_settings().replicate.key appears twice in the litellm_ai_handler.py file, which seems like a copy-paste error.
🔒 Security concerns	No

---

✨ Review tool usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

• When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

• With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Code Suggestions

Category	Suggestions
Maintainability	Standardize the naming convention for dictionary keys to maintain consistency. Consider using a consistent naming convention for the keys in the dictionary. The existing keys use a format that includes the model type, version, and date, while the new keys for Groq Llama3 do not follow this pattern. This can lead to confusion and errors in key usage elsewhere in the code. pr_agent/algo/init.py [34-35] -'groq/llama3-8b-8192': 8192, -'groq/llama3-70b-8192': 8192, +'groq/llama3-8b-20240308-v1': 8192, +'groq/llama3-70b-20240308-v1': 8192,
	Rename the attribute to groq_key for clarity and consistency. The attribute name api_key used for Groq might be too generic compared to other specific keys like anthropic_key, cohere_key, etc. Consider renaming it to groq_key to maintain consistency and clarity in attribute naming. pr_agent/algo/ai_handlers/litellm_ai_handler.py [56] -litellm.api_key = get_settings().groq.key +litellm.groq_key = get_settings().groq.key
Enhancement	Enhance the documentation with additional setup details and considerations. The documentation for setting up the Groq configuration should include more detailed steps or considerations, such as handling different environments or potential common issues. This will help users better understand and implement the configuration correctly. docs/docs/usage-guide/additional_configurations.md [128-138] -To use Llama3 model with Groq, for example, set: +To use Llama3 model with Groq, consider the following setup: [config] # in configuration.toml model = "llama3-70b-8192" model_turbo = "llama3-70b-8192" fallback_models = ["groq/llama3-70b-8192"] +# Ensure to handle different environments like development, testing, and production. [groq] # in .secrets.toml key = ... # your Groq api key +# Check for common issues like network timeouts or key permissions.
Best practice	Add a reminder comment to replace the API key placeholder. Add a comment next to the Groq key placeholder to remind users to replace the placeholder with their actual API key, as this is a common oversight that can lead to runtime errors. pr_agent/settings/.secrets_template.toml [33] [groq] -key = "" # Acquire through https://console.groq.com/keys +key = "" # Acquire through https://console.groq.com/keys. Replace this placeholder with your actual Groq API key.

---

✨ Improve tool usage guide:

---

Overview:
The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.

• When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

• With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

See the improve usage page for a comprehensive guide on using this tool.

[mrT23]

@randy-tsukemen were you able to run llama3 with groq on PR-Agent ?

[randy-tsukemen]

@mrT23
I was able to run Groq llama3 with the changes mentioned by this PR.
Groq does not need credit card information to generate API keys and has free quota everyday.