Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump gocloud.dev from 0.37.0 to 0.39.0 #2160

Merged
merged 1 commit into from
Sep 10, 2024

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Sep 9, 2024

Bumps gocloud.dev from 0.37.0 to 0.39.0.

Release notes

Sourced from gocloud.dev's releases.

v0.39.0

BREAKING CHANGE (AWS only, V1 vs V2 SDK)

Context: AWS has announced maintenance mode for the Go V1 SDK.

Go CDK has changed the default SDK for URLs across all modules except docstore/awsdynamodb to be V2 (previously you needed to add awssdk=v2 to the URL to get V2). Most URLs should continue to work, but in some cases you may need to add awssdk=v1 to force V1 explicitly.

Also, concrete type constructors (e.g., OpenBucket) for V1 (again, except docstore/awsdynamodb) have been marked deprecated; please migrate to using the V2 versions (e.g., OpenBucketV2).

Our tentative plan is to remove support for V1 in early 2025; please file a bug if you have concerns about that.

What's Changed

New Contributors

Full Changelog: google/go-cloud@v0.38.0...v0.39.0

v0.38.0

blob

pubsub

docstore

New Contributors

... (truncated)

Commits
  • 00a16e0 all: prerelease (#3469)
  • 8512f21 all: update deps (#3468)
  • 3d1f45c all: update to go version 1.23 (#3467)
  • 56e722d aws/all: BREAKING CHANGE Change the default for AWS URLs from V1 to V2. (#3465)
  • 526ff45 aws/all: Mark V1 constructors deprecated. (#3466)
  • d245f10 docstore/all: Add support for boolean filter (#3464)
  • 47ba1ac pubsub/awssnssqs: Fix race condition in new test, and call ack to avoid warni...
  • b70270a build(deps): bump the go_modules group across 2 directories with 1 update (#3...
  • 3929286 pubsub: tidy and cleanup (#3462)
  • 8796631 pubsub/all: Make batch request results independent (#3457)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.37.0 to 0.39.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](google/go-cloud@v0.37.0...v0.39.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file go Pull requests that update Go code labels Sep 9, 2024
@ChenYi015
Copy link
Contributor

/approve
/lgtm

Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ChenYi015

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit e58023b into master Sep 10, 2024
8 checks passed
@dependabot dependabot bot deleted the dependabot/go_modules/gocloud.dev-0.39.0 branch September 10, 2024 06:37
ChenYi015 pushed a commit to ChenYi015/spark-operator that referenced this pull request Sep 10, 2024
Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.37.0 to 0.39.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](google/go-cloud@v0.37.0...v0.39.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit e58023b)
Signed-off-by: Yi Chen <github@chenyicn.net>
google-oss-prow bot pushed a commit that referenced this pull request Sep 23, 2024
* Support gang scheduling with Yunikorn (#2107)

* Add Yunikorn scheduler and example

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add test cases

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add code comments

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add license comment

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Inline mergeNodeSelector

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Fix initial number implementation

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 8fcda12)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile for building sparkctl (#2119)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 4bc6e89)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: Add default values for namespaces to match usage descriptions  (#2128)

* fix: Add default values for namespaces to match usage descriptions

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>

* fix: remove incorrect cache settings

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>

---------

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>
Co-authored-by: pengfei4.li <pengfei4.li@ly.com>
(cherry picked from commit 52f818d)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Fix: Spark role binding did not render properly when setting spark service account name (#2135)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit a1a38ea)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Reintroduce option webhook.enable (#2142)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 9e88049)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Add default batch scheduler argument (#2143)

* Add default batch scheduler argument

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add helm unit test

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 9cc1c02)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: unable to set controller/webhook replicas to zero (#2147)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 1afa72e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Adding support for setting spark job namespaces to all namespaces (#2123)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit c93b0ec)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Support extended kube-scheduler as batch scheduler (#2136)

* Support coscheduling with kube-scheduler plugins

Signed-off-by: Yi Chen <github@chenyicn.net>

* Add example for using kube-schulder coscheduling

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit e8d3de9)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Run e2e tests on Kind (#2148)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit c810ece)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Set schedulerName to Yunikorn (#2153)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 62b4ca6)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Create role and rolebinding for controller/webhook in every spark job namespace if not watching all namespaces (#2129)

watching all namespaces

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 592b649)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Fix: e2e test failes due to webhook not ready (#2149)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit dee91ba)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Upgrade to Go 1.23.1 (#2155)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 10fcb8e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Upgrade to Spark 3.5.2 (#2154)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit e1b7a27)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump sigs.k8s.io/scheduler-plugins from 0.29.7 to 0.29.8 (#2159)

Bumps [sigs.k8s.io/scheduler-plugins](https://github.com/kubernetes-sigs/scheduler-plugins) from 0.29.7 to 0.29.8.
- [Release notes](https://github.com/kubernetes-sigs/scheduler-plugins/releases)
- [Changelog](https://github.com/kubernetes-sigs/scheduler-plugins/blob/master/RELEASE.md)
- [Commits](kubernetes-sigs/scheduler-plugins@v0.29.7...v0.29.8)

---
updated-dependencies:
- dependency-name: sigs.k8s.io/scheduler-plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 95d202e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* feat: support driver and executor pod use different priority (#2146)

* feat: support driver and executor pod use different priority

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: if *app.Spec.Driver.PriorityClassName and *app.Spec.Executor.PriorityClassName specifically defined, then can precedence over spec.batchSchedulerOptions.priorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: merge the logic of setPodPriorityClassName into addPriorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: support driver and executor pod use different priority

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: if *app.Spec.Driver.PriorityClassName and *app.Spec.Executor.PriorityClassName specifically defined, then can precedence over spec.batchSchedulerOptions.priorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: merge the logic of setPodPriorityClassName into addPriorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: add adjust pointer if is nil

Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: remove spec.batchSchedulerOptions.priorityClassName define , split driver and executor pod priorityClass

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: remove spec.batchSchedulerOptions.priorityClassName define , split driver and executor pod priorityClass

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: Optimize code to avoid null pointer exceptions

Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* fix: remove backup crd files

Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* fix: remove BatchSchedulerOptions.PriorityClassName test code

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* fix: add driver and executor pod priorityClassName test code

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

---------

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>
Co-authored-by: Kevin Wu <kevin.wu@momenta.ai>
(cherry picked from commit 6ae1b2f)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump gocloud.dev from 0.37.0 to 0.39.0 (#2160)

Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.37.0 to 0.39.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](google/go-cloud@v0.37.0...v0.39.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit e58023b)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Update e2e tests (#2161)

* Add sleep buffer to ensture the webhooks are ready before running the e2e tests

Signed-off-by: Yi Chen <github@chenyicn.net>

* Remove duplicate operator image build tasks

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update e2e tests

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update examples

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit e6a7805)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: webhook not working when settings spark job namespaces to empty (#2163)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 7785107)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: The logger had an odd number of arguments, making it panic (#2166)

Signed-off-by: tcassaert <tcassaert@inuits.eu>
(cherry picked from commit eb48b34)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Upgrade to Spark 3.5.2(#2012) (#2157)

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

---------

Signed-off-by: HyukSangCho <a01045542949@gmail.com>
(cherry picked from commit 9f0c08a)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Feature: Add pprof endpoint (#2164)

* add pprof support to the operator Controller Manager

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

* add pprof support to helm chart

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

---------

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
(cherry picked from commit 75b9266)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix the make kind-delete-custer to avoid accidental kubeconfig deletion (#2172)

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
(cherry picked from commit cbfefd5)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump github.com/aws/aws-sdk-go-v2/config from 1.27.27 to 1.27.33 (#2174)

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.27.27 to 1.27.33.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@config/v1.27.27...config/v1.27.33)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit b818332)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump helm.sh/helm/v3 from 3.15.3 to 3.16.1 (#2173)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.15.3 to 3.16.1.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.15.3...v3.16.1)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit f3f80d4)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Add specific error in log line when failed to create web UI service (#2170)

* Add specific error in log line when failed to create web UI service

Signed-off-by: tcassaert <tcassaert@inuits.eu>

* Update log to reflect correct resource that could not be created

Co-authored-by: Yi Chen <github@chenyicn.net>
Signed-off-by: tcassaert <tcassaert@protonmail.com>

---------

Signed-off-by: tcassaert <tcassaert@inuits.eu>
Signed-off-by: tcassaert <tcassaert@protonmail.com>
Co-authored-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit ed3226e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Account for spark.executor.pyspark.memory in Yunikorn gang scheduling (#2178)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit a2f71c6)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Fix: spark application does not respect time to live seconds (#2165)

* Add time to live seconds example spark application

Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: spark application does not respect time to live seconds

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit c855ee4)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Update release workflow and docs (#2121)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit bca6aa8)
Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: pengfei4.li <pengfei4.li@ly.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>
Signed-off-by: tcassaert <tcassaert@inuits.eu>
Signed-off-by: HyukSangCho <a01045542949@gmail.com>
Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
Signed-off-by: tcassaert <tcassaert@protonmail.com>
Co-authored-by: Jacob Salway <jacob.salway@gmail.com>
Co-authored-by: Neo <56439757+snappyyouth@users.noreply.github.com>
Co-authored-by: pengfei4.li <pengfei4.li@ly.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kevinz <ruoshuidba@gmail.com>
Co-authored-by: Kevin Wu <kevin.wu@momenta.ai>
Co-authored-by: tcassaert <tcassaert@protonmail.com>
Co-authored-by: ha2hi <56156892+ha2hi@users.noreply.github.com>
Co-authored-by: Sébastien Maintrot <3097030+ImpSy@users.noreply.github.com>
jbhalodia-slack pushed a commit to jbhalodia-slack/spark-operator that referenced this pull request Oct 4, 2024
* Support gang scheduling with Yunikorn (kubeflow#2107)

* Add Yunikorn scheduler and example

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add test cases

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add code comments

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add license comment

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Inline mergeNodeSelector

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Fix initial number implementation

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 8fcda12)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Update Makefile for building sparkctl (kubeflow#2119)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 4bc6e89)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: Add default values for namespaces to match usage descriptions  (kubeflow#2128)

* fix: Add default values for namespaces to match usage descriptions

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>

* fix: remove incorrect cache settings

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>

---------

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>
Co-authored-by: pengfei4.li <pengfei4.li@ly.com>
(cherry picked from commit 52f818d)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Fix: Spark role binding did not render properly when setting spark service account name (kubeflow#2135)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit a1a38ea)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Reintroduce option webhook.enable (kubeflow#2142)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 9e88049)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Add default batch scheduler argument (kubeflow#2143)

* Add default batch scheduler argument

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add helm unit test

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 9cc1c02)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: unable to set controller/webhook replicas to zero (kubeflow#2147)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 1afa72e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Adding support for setting spark job namespaces to all namespaces (kubeflow#2123)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit c93b0ec)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Support extended kube-scheduler as batch scheduler (kubeflow#2136)

* Support coscheduling with kube-scheduler plugins

Signed-off-by: Yi Chen <github@chenyicn.net>

* Add example for using kube-schulder coscheduling

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit e8d3de9)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Run e2e tests on Kind (kubeflow#2148)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit c810ece)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Set schedulerName to Yunikorn (kubeflow#2153)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 62b4ca6)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Create role and rolebinding for controller/webhook in every spark job namespace if not watching all namespaces (kubeflow#2129)

watching all namespaces

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 592b649)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Fix: e2e test failes due to webhook not ready (kubeflow#2149)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit dee91ba)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Upgrade to Go 1.23.1 (kubeflow#2155)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit 10fcb8e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Upgrade to Spark 3.5.2 (kubeflow#2154)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit e1b7a27)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump sigs.k8s.io/scheduler-plugins from 0.29.7 to 0.29.8 (kubeflow#2159)

Bumps [sigs.k8s.io/scheduler-plugins](https://github.com/kubernetes-sigs/scheduler-plugins) from 0.29.7 to 0.29.8.
- [Release notes](https://github.com/kubernetes-sigs/scheduler-plugins/releases)
- [Changelog](https://github.com/kubernetes-sigs/scheduler-plugins/blob/master/RELEASE.md)
- [Commits](kubernetes-sigs/scheduler-plugins@v0.29.7...v0.29.8)

---
updated-dependencies:
- dependency-name: sigs.k8s.io/scheduler-plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit 95d202e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* feat: support driver and executor pod use different priority (kubeflow#2146)

* feat: support driver and executor pod use different priority

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: if *app.Spec.Driver.PriorityClassName and *app.Spec.Executor.PriorityClassName specifically defined, then can precedence over spec.batchSchedulerOptions.priorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: merge the logic of setPodPriorityClassName into addPriorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: support driver and executor pod use different priority

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: if *app.Spec.Driver.PriorityClassName and *app.Spec.Executor.PriorityClassName specifically defined, then can precedence over spec.batchSchedulerOptions.priorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: merge the logic of setPodPriorityClassName into addPriorityClassName

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: add adjust pointer if is nil

Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* feat: remove spec.batchSchedulerOptions.priorityClassName define , split driver and executor pod priorityClass

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: remove spec.batchSchedulerOptions.priorityClassName define , split driver and executor pod priorityClass

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* feat: Optimize code to avoid null pointer exceptions

Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* fix: remove backup crd files

Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>

* fix: remove BatchSchedulerOptions.PriorityClassName test code

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

* fix: add driver and executor pod priorityClassName test code

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>

---------

Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>
Co-authored-by: Kevin Wu <kevin.wu@momenta.ai>
(cherry picked from commit 6ae1b2f)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump gocloud.dev from 0.37.0 to 0.39.0 (kubeflow#2160)

Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.37.0 to 0.39.0.
- [Release notes](https://github.com/google/go-cloud/releases)
- [Commits](google/go-cloud@v0.37.0...v0.39.0)

---
updated-dependencies:
- dependency-name: gocloud.dev
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit e58023b)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Update e2e tests (kubeflow#2161)

* Add sleep buffer to ensture the webhooks are ready before running the e2e tests

Signed-off-by: Yi Chen <github@chenyicn.net>

* Remove duplicate operator image build tasks

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update e2e tests

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update examples

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit e6a7805)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: webhook not working when settings spark job namespaces to empty (kubeflow#2163)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit 7785107)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: The logger had an odd number of arguments, making it panic (kubeflow#2166)

Signed-off-by: tcassaert <tcassaert@inuits.eu>
(cherry picked from commit eb48b34)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Upgrade to Spark 3.5.2(kubeflow#2012) (kubeflow#2157)

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

* Upgrade to Spark 3.5.2

Signed-off-by: HyukSangCho <a01045542949@gmail.com>

---------

Signed-off-by: HyukSangCho <a01045542949@gmail.com>
(cherry picked from commit 9f0c08a)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Feature: Add pprof endpoint (kubeflow#2164)

* add pprof support to the operator Controller Manager

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

* add pprof support to helm chart

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

---------

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
(cherry picked from commit 75b9266)
Signed-off-by: Yi Chen <github@chenyicn.net>

* fix the make kind-delete-custer to avoid accidental kubeconfig deletion (kubeflow#2172)

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
(cherry picked from commit cbfefd5)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump github.com/aws/aws-sdk-go-v2/config from 1.27.27 to 1.27.33 (kubeflow#2174)

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.27.27 to 1.27.33.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@config/v1.27.27...config/v1.27.33)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit b818332)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Bump helm.sh/helm/v3 from 3.15.3 to 3.16.1 (kubeflow#2173)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.15.3 to 3.16.1.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.15.3...v3.16.1)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit f3f80d4)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Add specific error in log line when failed to create web UI service (kubeflow#2170)

* Add specific error in log line when failed to create web UI service

Signed-off-by: tcassaert <tcassaert@inuits.eu>

* Update log to reflect correct resource that could not be created

Co-authored-by: Yi Chen <github@chenyicn.net>
Signed-off-by: tcassaert <tcassaert@protonmail.com>

---------

Signed-off-by: tcassaert <tcassaert@inuits.eu>
Signed-off-by: tcassaert <tcassaert@protonmail.com>
Co-authored-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit ed3226e)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Account for spark.executor.pyspark.memory in Yunikorn gang scheduling (kubeflow#2178)

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
(cherry picked from commit a2f71c6)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Fix: spark application does not respect time to live seconds (kubeflow#2165)

* Add time to live seconds example spark application

Signed-off-by: Yi Chen <github@chenyicn.net>

* fix: spark application does not respect time to live seconds

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit c855ee4)
Signed-off-by: Yi Chen <github@chenyicn.net>

* Update release workflow and docs (kubeflow#2121)

Signed-off-by: Yi Chen <github@chenyicn.net>
(cherry picked from commit bca6aa8)
Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
Signed-off-by: Yi Chen <github@chenyicn.net>
Signed-off-by: pengfei4.li <pengfei4.li@ly.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Kevin Wu <kevin.wu@momenta.ai>
Signed-off-by: Kevin.Wu <kevin.wu@momenta.ai>
Signed-off-by: tcassaert <tcassaert@inuits.eu>
Signed-off-by: HyukSangCho <a01045542949@gmail.com>
Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
Signed-off-by: tcassaert <tcassaert@protonmail.com>
Co-authored-by: Jacob Salway <jacob.salway@gmail.com>
Co-authored-by: Neo <56439757+snappyyouth@users.noreply.github.com>
Co-authored-by: pengfei4.li <pengfei4.li@ly.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kevinz <ruoshuidba@gmail.com>
Co-authored-by: Kevin Wu <kevin.wu@momenta.ai>
Co-authored-by: tcassaert <tcassaert@protonmail.com>
Co-authored-by: ha2hi <56156892+ha2hi@users.noreply.github.com>
Co-authored-by: Sébastien Maintrot <3097030+ImpSy@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved dependencies Pull requests that update a dependency file go Pull requests that update Go code lgtm size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant