Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the process to build api-docs, generate CRD manifests and code #2046

Merged
merged 9 commits into from
Jul 22, 2024

Conversation

ChenYi015
Copy link
Contributor

@ChenYi015 ChenYi015 commented Jun 5, 2024

Purpose of this PR

Proposed changes:

  • Update Makefile by adding some useful targets
  • Generate RBAC, CustomResourceDefinition and WebhookConfiguration manifests by make manifests, controller-gen will be automatically installed into bin directory if needed
  • Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations by make generate, controller-gen will be automatically installed into bin directory if needed
  • Update api-docs and related action workflow, make build-api-docs will automatically install gen-crd-api-reference-docs under bin directory and then used it to generate docs, no need to build docker image locally anymore.
  • Add bin/ to .gitignore

Change Category

Indicate the type of change by marking the applicable boxes:

  • Bugfix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that could affect existing functionality)
  • Documentation update

Rationale

Add some usefule targets to Makefile, the availabe targets are as follows:

$ make                              

Usage:
  make <target>

General
  help             Display this help.

Development
  manifests        Generate CustomResourceDefinition, RBAC and WebhookConfiguration manifests.
  generate         Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
  update-crd       Update CRD files in the Helm chart.
  clean            Clean up caches and output.
  go-fmt           Run go fmt against code.
  go-vet           Run go vet against code.
  lint             Run golangci-lint linter.
  lint-fix         Run golangci-lint linter and perform fixes.
  unit-test        Run go unit tests.
  e2e-test         Run go integration tests.

Build
  build-operator   Build spark-operator binary.
  build-sparkctl   Build sparkctl binary.
  install-sparkctl  Install sparkctl binary.
  clean-sparkctl   Clean sparkctl binary.
  build-api-docs   Build api documentaion.
  docker-build     Build docker image with the operator.
  docker-push      Push docker image with the operator.
  docker-buildx    Build and push docker image for the operator for cross-platform support.

Helm
  helm-unittest    Run Helm chart unittests.
  helm-lint        Run Helm chart lint test.
  helm-docs        Generates markdown documentation for helm charts from requirements and values files.

Deployment
  install          Install CRDs into the K8s cluster specified in ~/.kube/config.
  uninstall        Uninstall CRDs from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
  deploy           Deploy controller to the K8s cluster specified in ~/.kube/config.
  undeploy         Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.

Dependencies
  kustomize        Download kustomize locally if necessary.
  controller-gen   Download controller-gen locally if necessary.
  kind             Download kind locally if necessary.
  envtest          Download setup-envtest locally if necessary.
  golangci-lint    Download golangci-lint locally if necessary.
  gen-crd-api-reference-docs  Download gen-crd-api-reference-docs locally if necessary.

Checklist

Before submitting your PR, please review the following:

  • I have conducted a self-review of my own code.
  • I have updated documentation accordingly.
  • I have added tests that prove my changes are effective or that my feature works.
  • Existing unit tests pass locally with my changes.

Additional Notes

@ChenYi015
Copy link
Contributor Author

/assign @yuchaoran2011 @vara-bonthu

Copy link
Contributor

@vara-bonthu vara-bonthu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChenYi015 Great work on updating the process for building API documentation and generating CRD manifests.
This will streamline the workflow and improve maintainability.

PR looks good to me.

@yuchaoran2011

@ChenYi015
Copy link
Contributor Author

PTAL @yuchaoran2011, thanks.

@ChenYi015
Copy link
Contributor Author

@yuchaoran2011 Could you merge this PR first? This will make the development process much more easier.

Copy link
Contributor

@yuchaoran2011 yuchaoran2011 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat improvements! Common tasks will be much more convenient to perform. Need a chart version bump and then should be good to merge

Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Yi Chen <github@chenyicn.net>
@ChenYi015
Copy link
Contributor Author

Neat improvements! Common tasks will be much more convenient to perform. Need a chart version bump and then should be good to merge

@yuchaoran2011 Rebased and bumped the chart version.

Copy link
Contributor

@yuchaoran2011 yuchaoran2011 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@google-oss-prow google-oss-prow bot added the lgtm label Jul 22, 2024
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vara-bonthu, yuchaoran2011

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [vara-bonthu,yuchaoran2011]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 779ea3d into kubeflow:master Jul 22, 2024
7 checks passed
@ChenYi015 ChenYi015 deleted the dev branch July 22, 2024 06:03
jacobsalway pushed a commit to jacobsalway/spark-operator that referenced this pull request Jul 25, 2024
…kubeflow#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
ChenYi015 added a commit to ChenYi015/spark-operator that referenced this pull request Aug 1, 2024
…kubeflow#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 779ea3d)
google-oss-prow bot pushed a commit that referenced this pull request Aug 1, 2024
* Update helm docs (#2081)

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
(cherry picked from commit eca3fc8)

* Update the process to build api-docs, generate CRD manifests and code (#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 779ea3d)

* Add topologySpreadConstraints (#2091)

* Update README and documentation (#2047)

* Update docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Remove docs and update README

Signed-off-by: Yi Chen <github@chenyicn.net>

* Add link to monthly community meeting

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Add PodDisruptionBudget to chart (#2078)

* Add PodDisruptionBudget to chart

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Signed-off-by: Carlos Sánchez Páez <sanchezpaezcarlos33@gmail.com>
Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>

* PR comments

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>

---------

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Signed-off-by: Carlos Sánchez Páez <sanchezpaezcarlos33@gmail.com>
Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Set topologySpreadConstraints

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Update README and increase patch version

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Revert replicaCount change

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Update README after master merger

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Update README

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>
Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Signed-off-by: Carlos Sánchez Páez <sanchezpaezcarlos33@gmail.com>
Co-authored-by: Yi Chen <github@chenyicn.net>
Co-authored-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
(cherry picked from commit 4108f54)

* Use controller-runtime to reconsturct spark operator (#2072)

* Use controller-runtime to reconstruct spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm charts

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update examples

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 0dc641b)

---------

Co-authored-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Co-authored-by: jbhalodia-slack <jbhalodia@salesforce.com>
YanivKunda pushed a commit to YanivKunda/spark-operator that referenced this pull request Aug 5, 2024
…kubeflow#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
sigmarkarl pushed a commit to spotinst/spark-on-k8s-operator that referenced this pull request Aug 7, 2024
…kubeflow#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
Kevinz857 pushed a commit to Kevinz857/spark-operator that referenced this pull request Aug 27, 2024
…kubeflow#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
jbhalodia-slack added a commit to jbhalodia-slack/spark-operator that referenced this pull request Oct 4, 2024
…ubeflow#2108)

* Update helm docs (kubeflow#2081)

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
(cherry picked from commit eca3fc8)

* Update the process to build api-docs, generate CRD manifests and code (kubeflow#2046)

* Update .gitignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update .dockerignore

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the process to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update the workflow to generate api docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Use controller-gen to generate CRD and deep copy related methods

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm chart CRDs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update workflow for building spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update README.md

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 779ea3d)

* Add topologySpreadConstraints (kubeflow#2091)

* Update README and documentation (kubeflow#2047)

* Update docs

Signed-off-by: Yi Chen <github@chenyicn.net>

* Remove docs and update README

Signed-off-by: Yi Chen <github@chenyicn.net>

* Add link to monthly community meeting

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Add PodDisruptionBudget to chart (kubeflow#2078)

* Add PodDisruptionBudget to chart

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Signed-off-by: Carlos Sánchez Páez <sanchezpaezcarlos33@gmail.com>
Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>

* PR comments

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>

---------

Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Signed-off-by: Carlos Sánchez Páez <sanchezpaezcarlos33@gmail.com>
Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Set topologySpreadConstraints

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Update README and increase patch version

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Revert replicaCount change

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Update README after master merger

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

* Update README

Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: jbhalodia-slack <jbhalodia@salesforce.com>
Signed-off-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Signed-off-by: Carlos Sánchez Páez <sanchezpaezcarlos33@gmail.com>
Co-authored-by: Yi Chen <github@chenyicn.net>
Co-authored-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
(cherry picked from commit 4108f54)

* Use controller-runtime to reconsturct spark operator (kubeflow#2072)

* Use controller-runtime to reconstruct spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm charts

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update examples

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 0dc641b)

---------

Co-authored-by: Carlos Sánchez Páez <karlossanpa@gmail.com>
Co-authored-by: jbhalodia-slack <jbhalodia@salesforce.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants