Add NetworkPolicy Recommendation on Snowflake Backend

.github/workflows/python.yml

@@ -102,3 +102,42 @@ jobs:
         uses: codecov/codecov-action@v3
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
+
+  check-udf-changes:
+    name: Check whether udf tests need to be run based on diff
+    runs-on: [ubuntu-latest]
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 0
+    - uses: antrea-io/has-changes@v2
+      id: check_diff
+      with:
+        paths: snowflake/udf/*
+    outputs:
+      has_changes: ${{ steps.check_diff.outputs.has_changes }}
+
+  test-udf:
+      needs: check-udf-changes
+      if: ${{ needs.check-udf-changes.outputs.has_changes == 'yes' }}
+      name: Udf test
+      strategy:
+        matrix:
+          os: [ubuntu-latest]
+          python-version: ["3.7"]
+      runs-on: ${{ matrix.os }}
+      steps:
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v3
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Check-out code
+        uses: actions/checkout@v3
+      - name: Install dependencies
+        run: |
+          python -m pip install six psutil python-dateutil urllib3 requests pyyaml
+          wget https://downloads.antrea.io/artifacts/snowflake-udf/k8s-client-python-v24.2.0.zip
+          unzip k8s-client-python-v24.2.0.zip -d snowflake/udf/
+      - name: Run udf tests
+        run: |
+          make -C snowflake/udf check

snowflake/Makefile

@@ -1,7 +1,11 @@
 GO                 ?= go
 BINDIR := $(CURDIR)/bin
 
-all: bin
+all: udfs bin
+
+.PHONY: udfs
+udfs:
+	make -C udfs/udfs/
 
 .PHONY: bin
 bin:

snowflake/README.md

@@ -15,6 +15,7 @@
   - [Configure the Flow Aggregator in your cluster(s)](#configure-the-flow-aggregator-in-your-clusters)
 - [Clean up](#clean-up)
 - [Running applications](#running-applications)
+  - [NetworkPolicy Recommendation](#networkpolicy-recommendation)
 - [Network flow visibility with Grafana](#network-flow-visibility-with-grafana)
   - [Configure datasource](#configure-datasource)
   - [Deployments](#deployments)
@@ -139,8 +140,32 @@ Snowflake credentials are required.
 
 ## Running applications
 
-We are in the process of adding support for applications to Snowflake-powered
-Theia, starting with NetworkPolicy recommendation.
+### NetworkPolicy Recommendation
+
+NetworkPolicy Recommendation recommends the NetworkPolicy configuration
+to secure Kubernetes network and applications. It analyzes the network flows
+stored in the Snowflake database to generate
+[Kubernetes NetworkPolicies](
+https://kubernetes.io/docs/concepts/services-networking/network-policies/)
+or [Antrea NetworkPolicies](
+https://github.com/antrea-io/antrea/blob/main/docs/antrea-network-policy.md).
+
+```bash
+# make sure you have called onboard before running policy-recommendation
+./bin/theia-sf policy-recommendation --database-name <SNOWFLAKE DATABASE NAME> > recommended_policies.yml
+```
+
+Database name can be found in the output of the [onboard](#getting-started)
+command.
+
+NetworkPolicy Recommendation requires a Snowflake warehouse to execute and may
+take seconds to minutes depending on the number of flows. We recommend using a
+[Medium size warehouse](https://docs.snowflake.com/en/user-guide/warehouses-overview.html)
+if you are working on a big dataset. If no warehouse is provided by the
+`--warehouse-name` option, we will create a temporary X-Small size warehouse by
+default. Running NetworkPolicy Recommendation will consume Snowflake credits,
+the amount of which will depend on the size of the warehouse and the contents
+of the database.
 
 ## Network flow visibility with Grafana