v0.4.1 - New LLM Tool, pySigma version upgrade

What's Changed

• New LLM tool added to convert a SIEM/Product query into a Sigma Rule (a.k.a reverse conversion)
• Default LLM models have been updated from gpt-3.5-turbo to gpt-4o
• Rule Creation prompt has been updated
  • Ensures better rules are created when user asks about threat group, malware activity
  • Schema URL given to prompt and instructed to look it up if LLM is unsure of correct schema for rule output
    • The Sigma Schema is already provided in the prompt, but this just gives it all the context it would need if required
  • Created rules should now include the original author and related rule IDs if rules were used as context for creating the new rule. This is to ensure the detection rule license is enforced
• pySigma core version increased to v0.10.10. Backend and pipeline versions were increased to their maximum allowed versions for this pySigma version.

Upcoming

• pySigma will be updated to at least v0.11.3. Backends and pipelines will be updated to the latest allowed version with this change.
• This will also allow us to update langchain and the LLM libraries to the latest versions, due to a conflict with the packaging dependency pinned versions in langchain and pysigma that was fixed in pysigma 0.11.3.

Full Changelog: v0.3.0...v0.4.0

v0.3.0: OpenAI / LLM Support

With this release, we've added LLM / OpenAI functionality! Here's some of the highlights:

• Added an rule updater to download the latest SigmaHQ Rule release
• Added base LLM class to create embeddings from downloaded rules and store in a local VectorStore
• Added simple similarity searching for Sigma Rules in a VectorStore from user input
• Added a langchain Toolkit and Tools for use with a langchain Agent/bot to perform the following:
  • Automatically convert a Sigma Rule to any SigmAIQ supported backend, pipeline, and output format via user input
  • Automatically create brand new Sigma Rules based on a user's input and similar rules in the VectorStore

This is still very much a work in progress, but we are excited to share this with the community and keep working on its development.

For more information, please see the LLM specific README here

v0.2.4

• Fixed improper pipeline creation when setting new pipeline in created SigmAIQBackend object

v0.2.3

• Significantly improved the performance of create_all_and_translate() from SigmAIQBackend
• Added optional filter to exclude specific backends from create_all_and_translate()
• Added black to dev dependencies
• Added new util to automatically create a SigmaRule or SigmaCollection object from one of the following types:
  • A SigmaRule or SigmaCollection (just returns the object)
  • A str consisting of valid Sigma rule YAML
  • A dict consisting of valid Sigma rule JSON
  • A list containing any of the above types
• Formatted code base with black

v0.2.2

Fixed pysigma-backend-qradar-aql pinned version causing errors

v0.2.1

• Pinned certifi version to 2023.07.22 to fix CVE-2023-37920

v0.2.0

New Backend

• Added support for the Cortex XDR Backend!

Updated Backend Versions

• Updated the following backends to the latest version
  • pysigma-backend-carbonblack: v0.1.2 -> v0.1.4
  • pysigma-backend-elasticsearch: v1.0.3 -> v1.0.5
  • pysigma-backend-qradar-aql: v0.1.3 -> v0.1.4
  • pysigma-backend-sentinelone: v0.1.1 -> v0.1.2

Bugfixes

• Fixed incorrect relative path of Splunk ES Correlation Search template
  • Custom output format "stanza" now works as intended, and will generated a savedsearches.conf file for a correlation search based on the output of the Splunk backend and Sigma Rule description/tags.

Initial Release

Initial pre-release