community[minor]: [Pebblo] Enhance PebbloSafeLoader to take anonymize flag #26812
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
the
size:M
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
dosubot
bot
added
community
ccurme
reviewed
Sep 24, 2024
| "source": [ | ||
| "### Anonymize the snippets to redact all PII details\n", | ||
| "\n", | ||
| "Set `anonymize_snippets` to `True` to anonymize all personally identifiable information (PII) from the snippets going into VectorDB and the generated reports." |
There was a problem hiding this comment.
Is the recall on identifying PII 100%? If not might be worth a clarification or link to other documentation to set expectations.
There was a problem hiding this comment.
Thank you for your feedback! I’ll add a link to the Pebblo classifier documentation to clarify the recall on identifying PII
There was a problem hiding this comment.
@ccurme I’ve added a note and a link to our docs. We’re planning to update it with more details soon. Please review.
Raj725
force-pushed
the
pebblo/anonymize-snippets
branch
from
0eb8f33
to
6eeadb8
Compare
Sheepsta300
pushed a commit
to Sheepsta300/langchain
that referenced
this pull request
Oct 1, 2024
… flag (langchain-ai#26812) - **Description:** The flag is named `anonymize_snippets`. When set to true, the Pebblo server will anonymize snippets by redacting all personally identifiable information (PII) from the snippets going into VectorDB and the generated reports - **Issue:** NA - **Dependencies:** NA - **docs**: Updated
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
anonymize_snippets. When set to true, the Pebblo server will anonymize snippets by redacting all personally identifiable information (PII) from the snippets going into VectorDB and the generated reports