Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG][data-prepper] Chart values should not have a default pipeline defined #624

Open
janhoy opened this issue Dec 4, 2024 · 3 comments · May be fixed by #634
Open

[BUG][data-prepper] Chart values should not have a default pipeline defined #624

janhoy opened this issue Dec 4, 2024 · 3 comments · May be fixed by #634
Labels
bug Something isn't working

Comments

@janhoy
Copy link

janhoy commented Dec 4, 2024

Describe the bug
The default values file has a demo pipeine defined. Even if you in your own values file define a my-pipeline, the demo simple-sample-pipeline will be deployed alongside it.

To Reproduce
Steps to reproduce the behavior:

  1. Create a simple myvalues.yaml file with a pipelineConfig.config.my-pipeline defined
  2. Install chart with -f myvalues.yaml
  3. Look at the foo-data-prepper-pipeline secret - it contains both pipelines.

Expected behavior
You'd expect that the pipeline you configure is the only one installed.

You can achieve a blank slate today by using the existsingSecret property, but for simple pipelines the convenience of defining it in values file should achieve the same.

Chart Name
data-prepper

Screenshots
Skjermbilde 2024-12-04 kl  11 21 49

Host/Environment (please complete the following information):

  • Helm Version: 3.16.3
  • Kubernetes Version: 1.30.2

Additional context
Proposed fix is to ship chart without a configured pipeline. Alternatively use a noop pipeline that does nothing, to make the prepper not crash. Instead, ship an examples/ folder and move the simple-sample-pipeline example to a separate values file in that folder so users can do helm install myprepper opensearch/data-prepper -f example-values.yaml to test.

@prudhvigodithi
Copy link
Member

[Triage]
I agree with you @janhoy can you start by commenting https://github.com/opensearch-project/helm-charts/blob/main/charts/data-prepper/values.yaml#L108 in the same values file, so that users can use that as example but actually would not be installed.
Thank you
@peterzhuamazon @TheAlgo

@prudhvigodithi prudhvigodithi removed the untriaged Issues that have not yet been triaged label Dec 5, 2024
@janhoy
Copy link
Author

janhoy commented Dec 17, 2024

We could comment the pipeline, but that would leave an empty config and prevent the prepper from starting at all after a helm install without parameters. Perhaps a way to achieve a dead simple working demo without providing your own pipeline could be to leave pipelineConfig.config empty in values, but provide a new value that people could set to enable a demo pipeline: helm install myPrepper opensearch/data-prepper --set pipelineConfig.demoPipeline=true. That would auto configure a demo pipeline in secret.yaml.

@janhoy
Copy link
Author

janhoy commented Dec 18, 2024

See the pull requewst implementing my proposal. This is a cleaner way to enable demo pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: 🆕 New
2 participants