Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

Support Enable/Disable NiFi Zookeeper client Zookeeper Ensemble Tracker #294

Closed
wants to merge 19 commits into from
Closed

Support Enable/Disable NiFi Zookeeper client Zookeeper Ensemble Tracker #294

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
253ad43
Disable zkClientEnsembleTracker
nathluu Apr 25, 2023
9bf4698
Update s2s bravo flow.xml
nathluu May 1, 2023
2a58c3d
Update s2s test to use cmctl from cert-manager instead of jetstack
nathluu May 1, 2023
f6a84e7
Change cert-manager installation
nathluu May 1, 2023
9d34bed
Fix s2s test (#
nathluu May 4, 2023
eaed349
Update s2s test
nathluu May 19, 2023
944db6d
Add session affinity FAQ (#299)
cf250024 Jun 9, 2023
f296a8e
feature/en289 Add a one-volume option for persistence using subPath (…
emrge-michaeld Jun 15, 2023
09dac85
Add auth.oidc.preferredJwsalgorithm param (#306)
happy-code-com Aug 8, 2023
f851d51
Add bootstrap options (#308)
combineads Aug 11, 2023
219b17e
relaunch tests
banzo Oct 2, 2023
3519298
Disable zkClientEnsembleTracker
nathluu Apr 25, 2023
020c282
Update s2s bravo flow.xml
nathluu May 1, 2023
09953ac
Update s2s test to use cmctl from cert-manager instead of jetstack
nathluu May 1, 2023
a3f7f5f
Change cert-manager installation
nathluu May 1, 2023
c003de2
Fix s2s test (#
nathluu May 4, 2023
317862a
Update s2s test
nathluu May 19, 2023
ea0c9e3
relaunch tests
banzo Oct 2, 2023
c0d0fb0
Merge branch 'nathluu/zkclient-ensemble-tracker-disable' of github.co…
nathluu Nov 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/test-site-to-site.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,12 +87,15 @@ jobs:
- name: Confirm certificate rotation
run: |
OLDCERTSTART=$(kubectl -n alpha exec nifi-0 -c server -- curl -kv https://localhost:8443 2>&1 | /bin/grep start)
echo Certificate "$OLDCERTSTART"
echo Old Certificate "$OLDCERTSTART"
cmctl -n alpha renew nifi-0
kubectl -n alpha rollout restart statefulset/nifi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This defeats the purpose of the test, which was to confirm that NiFi is automatically detecting the certificate has changed and restarting the TLS modules. If we want to disable this test because NiFi is broken, then I would recommend commenting it out with a comment rather than forcing a restart.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a TODO to remove this block of code when NIFI issue is fixed

kubectl -n alpha rollout status --watch statefulset/nifi --timeout=5m
sleep 5m
for n in [ 0 1 2 3 4 5 6 7 8 9 ]
do
NEWCERTSTART=$(kubectl -n alpha exec nifi-0 -c server -- curl -kv https://localhost:8443 2>&1 | /bin/grep start)
echo Certificate "$NEWCERTSTART"
echo New Certificate "$NEWCERTSTART"
if [ \""$OLDCERTSTART"\" != \""$NEWCERTSTART"\" ]
then
exit 0
Expand Down
4 changes: 2 additions & 2 deletions Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
apiVersion: v2
name: nifi
version: 1.1.4
appVersion: 1.16.3
version: 1.1.5
appVersion: 1.20.0
description: Apache NiFi is a software project from the Apache Software Foundation designed to automate the flow of data between software systems.
keywords:
- nifi
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ The following table lists the configurable parameters of the nifi chart and the
| `replicaCount` | Number of nifi nodes | `1` |
| **Image** |
| `image.repository` | nifi Image name | `apache/nifi` |
| `image.tag` | nifi Image tag | `1.16.3` |
| `image.tag` | nifi Image tag | `1.20.0` |
| `image.pullPolicy` | nifi Image pull policy | `IfNotPresent` |
| `image.pullSecret` | nifi Image pull secret | `nil` |
| **SecurityContext** |
Expand Down
1 change: 1 addition & 0 deletions configs/nifi.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ nifi.cluster.flow.election.max.wait.time=1 mins
nifi.cluster.flow.election.max.candidates=

# zookeeper properties, used for cluster management #
nifi.zookeeper.client.ensembleTracker={{.Values.properties.zkClientEnsembleTraker}}
nifi.zookeeper.connect.string=
nifi.zookeeper.connect.timeout=3 secs
nifi.zookeeper.session.timeout=3 secs
Expand Down
2 changes: 1 addition & 1 deletion doc/INSTALLATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ helm dep up
```
2. **Set a sensitiveKey**

In 1.16.3 version, Nifi needs a sensitiveKey to encrypt sensitive information. This key can be setted in the `values.yaml` file:
In 1.20.0 version, Nifi needs a sensitiveKey to encrypt sensitive information. This key can be setted in the `values.yaml` file:

````
properties:
Expand Down
1 change: 1 addition & 0 deletions templates/statefulset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,7 @@ spec:
fi
{{ end }}

prop_replace nifi.ui.banner.text $(hostname -s)
prop_replace nifi.remote.input.host ${FQDN}
prop_replace nifi.cluster.node.address ${FQDN}
prop_replace nifi.zookeeper.connect.string ${NIFI_ZOOKEEPER_CONNECT_STRING}
Expand Down
6 changes: 3 additions & 3 deletions tests/06-alpha.flow.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-standard-nar</artifact>
<version>1.16.3</version>
<version>1.20.0</version>
</bundle>
<maxConcurrentTasks>1</maxConcurrentTasks>
<schedulingPeriod>1 sec</schedulingPeriod>
Expand Down Expand Up @@ -117,7 +117,7 @@
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-ssl-context-service-nar</artifact>
<version>1.16.3</version>
<version>1.20.0</version>
</bundle>
<enabled>true</enabled>
<property>
Expand Down Expand Up @@ -162,7 +162,7 @@
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-site-to-site-reporting-nar</artifact>
<version>1.16.3</version>
<version>1.20.0</version>
</bundle>
<schedulingPeriod>5 sec</schedulingPeriod>
<scheduledState>RUNNING</scheduledState>
Expand Down
6 changes: 3 additions & 3 deletions tests/06-bravo.flow.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-update-attribute-nar</artifact>
<version>1.15.2</version>
<version>1.20.0</version>
</bundle>
<maxConcurrentTasks>1</maxConcurrentTasks>
<schedulingPeriod>0 sec</schedulingPeriod>
Expand Down Expand Up @@ -62,7 +62,7 @@
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-update-attribute-nar</artifact>
<version>1.15.2</version>
<version>1.20.0</version>
</bundle>
<maxConcurrentTasks>1</maxConcurrentTasks>
<schedulingPeriod>0 sec</schedulingPeriod>
Expand Down Expand Up @@ -159,7 +159,7 @@
<bundle>
<group>org.apache.nifi</group>
<artifact>nifi-prometheus-nar</artifact>
<version>1.15.2</version>
<version>1.20.0</version>
</bundle>
<schedulingPeriod>5 sec</schedulingPeriod>
<scheduledState>RUNNING</scheduledState>
Expand Down
3 changes: 2 additions & 1 deletion values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ replicaCount: 1
##
image:
repository: apache/nifi
tag: "1.16.3"
tag: "1.20.0"
pullPolicy: "IfNotPresent"

## Optionally specify an imagePullSecret.
Expand Down Expand Up @@ -94,6 +94,7 @@ properties:
httpsPort: 8443
webProxyHost: # <clusterIP>:<NodePort> (If Nifi service is NodePort or LoadBalancer)
clusterPort: 6007
zkClientEnsembleTraker: false # https://issues.apache.org/jira/browse/NIFI-10481
provenanceStorage: "8 GB"
provenanceMaxStorageTime: "10 days"
siteToSite:
Expand Down