Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DP-8085 - Migrate to Semaphore self-hosted agent #9

Merged
merged 1 commit into from
Aug 2, 2022
Merged

DP-8085 - Migrate to Semaphore self-hosted agent #9

merged 1 commit into from
Aug 2, 2022

Conversation

nlou9
Copy link
Member

@nlou9 nlou9 commented Jul 8, 2022

Background

In the current Semaphore cloud offering, all of the Semaphore build nodes are controlled by Semaphore. Thus, we are unable to control what resources they can access.
It also represents security concerns because Semaphore has access to what is on those nodes, including our credentials from Vault.
The DevProd team will be migrating the existing SemaphoreCI pipelines using linux machines to start using self-hosted Semaphore build agents.
Deadline to migrate the SemaphoreCI jobs to self hosted agents is July 22nd, 2022
The DevProd team will start creating PRs to migrate your existing Semaphore jobs to the self-hosted agents by July 10, 2022.

This migrates to Semaphore self-hosted agent

Changelog

Details on the self-hosted agent host configuration are described here : https://confluentinc.atlassian.net/wiki/spaces/TOOLS/pages/2820542346/Semaphore+Self+Hosted+Agents

Migration Changes

This automated pull request changes

  • Pipeline YAML files under .semaphore folder

Actions

  • If you are happy with the changes, feel free to merge.
  • If you have concerns about the changes, comment on this pull request, tagging the Developer Productivity team (@confluentinc/tools).
  • If you happen to have time to look at any of the failures and can help us out, that would be greatly appreciated.
  • If the CI does not pass, we may fix some simpler issues on this pull request, but if there is a deeper problem associated with the migration, we may create a task for your team to fix.

@nlou9 nlou9 requested a review from a team as a code owner July 8, 2022 18:38
@xvrl xvrl merged commit 0c355b1 into cc-druid-operator Aug 2, 2022
@xvrl xvrl deleted the DP-8085-Migrate-Semaphore-Agent branch August 2, 2022 12:40
CodingParsley pushed a commit that referenced this pull request Dec 14, 2022
@nlou9 @CodingParsley
DP-8085 - Migrate to Sempahore self-hosted agent (#9)
c5f2785
@CodingParsley CodingParsley mentioned this pull request Dec 19, 2022
Merged
4 tasks
CodingParsley pushed a commit that referenced this pull request Jan 6, 2023
@nlou9 @CodingParsley
DP-8085 - Migrate to Sempahore self-hosted agent (#9)
aae5c9e
CodingParsley added a commit that referenced this pull request Jan 6, 2023
@CodingParsley @piotrkmita
@AdheipSingh @achetronic @shrutimantri @harinirajendran @RoelofKuijpers @youngwookim @vladislavPV @zhangluva @cintoSunny @lyoung-confluent @freyayunfu @nlou9 @cchristous @ConfluentJenkins
OBSDATA-492: Sync with upstream druid-operator (#16)
4595a63 
* Restrict K8s role definitions to actually needed roles (druid-io#226)

* PK | druid-io#255 | Restrict K8s role definitions to actually needed roles

* PK | druid-io#255 | Restrict K8s role definitions to actually needed roles - templates/

* PK | druid-io#255 | Add patch to event and newline at the end of the file

* PK | druid-io#255 | Add patch to event to templates

Co-authored-by: Piotr Kmita <piotr.kmita@zalando.de>

* Introduce State Driven Status, use record pkg for event and clean logging (druid-io#223)

* status v1, records

* use druid constructor in main.go

* fix smoke tests

* fix comments

* update delete event, status

* push emitter changes

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* update to go.mod to k8s 1.22 (druid-io#237)

* update to k8s 1.22

* update go.mod

* use 1.19.2 for suit_test

* revert yaml change

* update identation/spaces in tiny

* update identation/spaces in tiny

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* Create watched_namespace.yaml (druid-io#248)

* Create watched_namespace.yaml

Create namespace that is being watched by Druid when declared and not being the default

* Update watched_namespace.yaml

Add some clarifications on comments

* Update watched_namespace.yaml

Fix to create all required namespaces by WATCH_NAMESPACE env var (comma separated list. This is intended to handle this future feature) only when they are not created yet

* Fix Helm Chart type error  (druid-io#253)

* run helm lint github actions

* add helm lint, template in CI

* Fix76: Scale STS / PVC vertically to support size resizing of druid nodes (druid-io#97)

* add storage class expansion check, generation check, refactor

* add flag for scalePvcSts

* add component label, update crd

* fix unit tests

* identation in testdata

* Fixing the image tag to be pulled for druid-operator (druid-io#259)

* the image tag fixed to druidio/druid-operator:latest

* Update Druid Operator Documentation (druid-io#255)

* update docs

* update typos

* update docs

* udpate docs

* Adding support for custom metricDimensions.json for statsd metrics emitter (druid-io#250)

* Adding support for custom metricDimensions.json.

* Addressing Adheip's comments.

* Fixing test failure.

* add patch to pvc rbac (druid-io#261)

* add support for druid 0.22.1 and zookeeper 3.7.0 (druid-io#267)

* add support for druid 0.22.1 and zookeeper 3.7.0

* Update tiny-cluster-zk.yaml

change ZooKeeper version from 3.7 to 3.7.0

* update appVersion to 0.0.8-release (druid-io#238)

Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>

* Fix-214-Merge-Labels (druid-io#273)

* add health checks to operator manage (druid-io#278)

* safe guard pod pending deletion of pvc (druid-io#289)

* make nodeselector consistent (druid-io#284)

* make nodeselector consitent

* update handler

* fix revert, scale down pods replicas to zero (druid-io#283)

* fix typo for startupProbe (druid-io#282)

* Bump up chart version to 0.1.2 (druid-io#271) (druid-io#272)

* Watch Multiple Namespaces (druid-io#240)

* WIP: on druid k8s extension, do, and watch ns

* support for multiple namespaces to watch

* support for multiple namespaces to watch

* fix typo

* fix identation

* Release 0.0.9 - Go Mod and Dockerfile Update (druid-io#291)

* go modules update

* add icon

* update dockerfile

* fix tests

* update helm version

* max reconciles workerqueue (druid-io#299)

* emit detect underlying object type on CRUD (druid-io#300)

* make crd optional (druid-io#307)

* multiple watched namespaces (druid-io#309)

* Sidecar container spec for druid (druid-io#296)

* Sidecar container spec for druid

* Fix PR comments on additionalcontainer

* Add envFrom and add comments

* Add example spec to example.md

* Add multi container deployment to features.md

Co-authored-by: cinto <cinto@apple.com>

* update makefile, tests, controller-gen (druid-io#315)

* update make

* update makefile

* Fix-312-Run Test on CI, Controller Gen Version Update (druid-io#319)

* update fixes

* add more tests

* fix test kubebuilder

* seperate build

* revert build

Co-authored-by: AdheipSingh <adheips1222@gmail.com>

* Adding minikube-setup instructions.

Update tiny-cluster.yaml to make it work in minikube.

Fixing operator startup bug.

Miscellaneous fixes to make local minikube setup work.

Fixing MM readiness probe and steps for minikube ingress-dns issue for local minikube setup.

* semaphore

path

generate

fix format

init-ci

add ci bin

path

kubebuilder move

dummy change

* Add command args to container creation (#6)

* go/codeowners: Generate CODEOWNERS [ci skip] (#7)

* [METRICS-4348] update obs-data team as codeownderswq (#8)

* [METRICS-4487] add obs-oncall as codeowners (#10)

* DP-8085 - Migrate to Sempahore self-hosted agent (#9)

* DP-9370 - Migrate to Semaphore self-hosted agent (#15)

* chore: update repo semaphore project

* Update service.yml (#17)

Update semaphore job commands and go version

update build triggers

update semaphore whitelist

Update project.yml

* update CI 

* nodespec should take precedence (#13)

Co-authored-by: piotrkmita <43473995+piotrkmita@users.noreply.github.com>
Co-authored-by: Piotr Kmita <piotr.kmita@zalando.de>
Co-authored-by: AdheipSingh <34169002+AdheipSingh@users.noreply.github.com>
Co-authored-by: AdheipSingh <adheip.singh@rilldata.com>
Co-authored-by: Alby Hernández <61636487+achetronic@users.noreply.github.com>
Co-authored-by: shrutimantri <shruti1810@gmail.com>
Co-authored-by: Harini Rajendran <harini.rajendran@yahoo.com>
Co-authored-by: RoelofKuijpers <roelof@datanetic.com>
Co-authored-by: Youngwoo Kim <ywkim@apache.org>
Co-authored-by: Vladislav <vladislavPV@users.noreply.github.com>
Co-authored-by: Zhang Lu <91473238+zhangluva@users.noreply.github.com>
Co-authored-by: cintoSunny <67714887+cintoSunny@users.noreply.github.com>
Co-authored-by: cinto <cinto@apple.com>
Co-authored-by: AdheipSingh <adheips1222@gmail.com>
Co-authored-by: Harini Rajendran <hrajendran@confluent.io>
Co-authored-by: Luke Young <91491244+lyoung-confluent@users.noreply.github.com>
Co-authored-by: Yun Fu <fuyun12345@gmail.com>
Co-authored-by: nlou9 <39046184+nlou9@users.noreply.github.com>
Co-authored-by: Corey Christous <cchristous@gmail.com>
Co-authored-by: Confluent Jenkins Bot <jenkins@confluent.io>
saithal-confluent pushed a commit that referenced this pull request Apr 6, 2023
@AdheipSingh
Merge pull request #9 from jwitko/helm/allow-keeping-crds
3f9c5e1 
helm: Keep CRD by default so druid clusters are not automatically rem…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xvrl xvrl xvrl approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nlou9 @xvrl