Feature/enhance safety tests with promptguard #1119
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chakravarthik27
changed the title
This commit refactors the PromptGuard class in the modelhandler/promptguard.py module. The changes include: - Simplifying the initialization process by using a singleton pattern - Loading the model and tokenizer from Hugging Face - Preprocessing the input text to remove spaces and mitigate prompt injection tactics - Calculating class probabilities for a single or batch of texts - Adding methods to get jailbreak scores and indirect injection scores for a single input text or a batch of texts - Processing texts in batches to improve efficiency The commit also includes changes in the safety.py module: - Importing the PromptGuard class from the modelhandler/promptguard.py module - Replacing the pipeline usage with the PromptGuard class to get indirect injection scores Lastly, the commit includes changes in the output.py and sample.py modules: - Adding a greater than or equal to comparison method in the MaxScoreOutput class - Updating the comparison method in the QASample class to use the new comparison method in MaxScoreOutput
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.