This document lists the Github repositories the Garden team has been responsible for. The document does not cover all [repositories](https://github.com/cloudfoundry?q=garden&type=&language=&sort=) containing `garden` in their name as we have lost our tribal knowledge on some of them. Probably they have been used in the past. # The Garden Code ## garden-runc-release https://github.com/cloudfoundry/garden-runc-release is the bosh release of Garden and the starting point of every development task. It contains: - bosh jobs - bosh packages - Garden submodules - utilities used by the bosh jobs, such as `greenskeeper` and `thresholder` - test utilities Garden bits are placed in different git repositories and they are submoduled here. After cloning the `garden-runc-release` repository do make sure to update its submodules to get the complete Garden code. Garden depends on external components that did not support go modules at the time of Garden adopting them. That is why they are submoduled under `src/gopath/src` and the `GO_PATH` environment variable [is set](https://github.com/cloudfoundry/garden-runc-release/blob/e9e37668d065d53876e54e75d79cc8457c1fe1a8/.envrc#L1) to `src/gopath` in order regular submodules to be built against the submoduled dependencies. `garden-runc-release` submodules are listed below ### dontpanic https://github.com/cloudfoundry/dontpanic is the Garden diagnostics tool, refer to [[Don't panic!]] for more details ### garden https://github.com/cloudfoundry/garden is the Garden server, the Garden API and the Garden client ### garden-integration-tests https://github.com/cloudfoundry/garden-integration-tests are Garden's acceptance tests (GATs). The name of the repository is a bit unfortunate as those are acceptance, not integration tests ### GrootFS https://github.com/cloudfoundry/grootfs is the container filesystem management utility ### guardian https://github.com/cloudfoundry/guardian is Guardian, the thing that makes Garden run ### idmapper https://github.com/cloudfoundry/idmapper contains a couple of small utilities: - `newuidmap` creates the `/proc/[pid]/uid_map` that map user IDs in the user namespace of the process specified by `pid`. Refer to the [user_namespaces](https://man7.org/linux/man-pages/man7/user_namespaces.7.html) man page for details - `newgidmap` creates the `/proc/[pid]/gid_map` that map group IDs in the user namespace of the process specified by `pid`. Refer to the [user_namespaces](https://man7.org/linux/man-pages/man7/user_namespaces.7.html) man page for details - `maximus` returns the ID of the user with the maximum user ID possible, hence the name. The maximum user ID differs on different systems, hence the need of such an utility. ### netplugin-shim https://github.com/cloudfoundry/netplugin-shim is the network plugin shim server and client. Only used for the experimental [bpm-enabled/rotless deployments](https://github.com/cloudfoundry/cf-deployment/blob/ca5cbab2b9af288cf9c54d9ce13dceeb428fa63c/operations/experimental/rootless-containers.yml#L16-L18) ## Garden Performance Acceptance Tests (GPATs) https://github.com/cloudfoundry/garden-performance-acceptance-tests contains the performance acceptance tests for Garden. The main [performance test suite](https://github.com/cloudfoundry/garden-performance-acceptance-tests/blob/bb9a3d891df70e02a4174e2089a74db0872e3609/create_test.go#L23) asseerting that container creation should take less than 1.5 seconds. ## The CPU Entitlement CF CLI plugin https://github.com/cloudfoundry/cpu-entitlement-plugin a CF CLI plugin to monitor CPU entitlement usage of running apps ## Groot https://github.com/cloudfoundry/groot is a library which can be used to build custom image plugins for Garden. For example, [groot-windows](https://github.com/cloudfoundry/groot-windows) uses that library to implement an image plugin for Windows. The code contained in this library is a shameless copy-paste from [GrootFS](https://github.com/cloudfoundry/grootfs), we had the idea to make GrootFS to use Groot but that never happened... shame... ## Garden dotfiles https://github.com/cloudfoundry/garden-dotfiles contains scripts for setting up a developer workstation ## garden-wiki https://github.com/cloudfoundry/garden-wiki is a wiki page containing ancient Garden legacy (some bits might be outdated). Contains interesting articles. # Pushing changes to the Garden code (i.e. `garden-runc-release` and its submodules) Once you have pulled `garden-runc-release` and updated its submodule, you can start implementing new stuff. Pushing stuff however obeys certain rules: 1. Garden submodules (see above) are developed in the `master` branch 1. We **do not push changes in dependencies** (such as `containerd` under the `src/gopath/src/github.com/containerd/containerd` path). You can implement an experimental change to see how it would work but you do not push it. Instead, we are only updating the version of the dependecy from the upstream. If you want to contribute a change to that dependency, you should open a pull request there and wait for a new release which you can bump to. 1. You **MUST NOT** push to the `garden-runc-release` `master` branch, this is up to CI to do that. You can only push to the default `develop` branch or `wip` branches. 1. Pushing is a multi-step process. Provided that you are happy with a change and all tests are green: 1. First, you commit you change in the submodule (note that bumping a dependency is also a commit) but **DO NOT** push it yet 2. Then, you run `./scripts/bump` from `~/workspace/garden-runc-release` in the `develop` branch. The script would pick up the new commits in the submodules and create a bump commit in the release referencing the submodules' new commits 3. Finally, you run `./scripts/gotta-push-em-all` from `~/workspace/garden-runc-release` in the `develop` branch. The script will recursively push the new commits in the submodules and the bump commit in the release. If you do not follow the procedure above and push changes to the submodules without the bump commit, CI will not pick them up. If you only push the bump commit without the commits in the submodules, CI would not be able to find them and would eventually fail. # Infrastructure repositories ## garden-ci https://github.com/cloudfoundry/garden-ci is the only private repository. It contains our CI infrastructure and test deployments. It submodules `bosh-deployment`, `cf-deployment`, `concourse-deployment` under the `modules` subdirectory. - `./ci` - contains the definition of the Garden CI pipeline - `./directors` - contains the state of all the bosh directors we use and their deployments - `./modules` - bosh deployments submodules. Scripts that create test [sandbox enritonments](./Sandbox-Environment.md) and update concourse use the versions being checked out there - `./ops` - various bosh ops files we found useful over the years - `./scripts` - various helper scripts ## garden-dockerfiles https://github.com/cloudfoundry/garden-dockerfiles contains various docker images used in CI or tests. The most notable image is `garden-ci` which is used by all CI jobs. ## Masters of cats https://github.com/masters-of-cats/ is an organisation where Garden puts repositories for experiments, forks, custom tools or random stuff. Initially the repository contained the `master` branch of [CATs](https://github.com/cloudfoundry/cf-acceptance-tests) (hence the name). The organisation is owned by @callisto13, an ex-Gardener, we are all at her mercy here :). Below are repostiries for tools used in our infrastructure - https://github.com/masters-of-cats/a-new-hope contains vim configurations we use for Garden daily work - https://github.com/masters-of-cats/concourse-flake-hunter is a tool to search for failed jobs on Concourse by a search string, useful to see whether an error has occured before - https://github.com/masters-of-cats/test-log-emitter - an http server that emits metrics to loggregator when requested, used in test deployments - https://github.com/masters-of-cats/test-log-emitter-release - a bosh release for https://github.com/masters-of-cats/test-log-emitter - https://github.com/masters-of-cats/dependachore - a GCP function that transforms issues created out of [dependabot](https://github.com/dependabot) PRs into Garden backlog chores (a.k.a. dependachores). We needed this tool because the anchor got bored to do that manually. - https://github.com/masters-of-cats/flightattendant - a small tool to check whether you are logged into the CI via checking whether you have a CI token. Used when running tests so that you are asked to log into the CI only when needed - https://github.com/masters-of-cats/vantablackbox-release - a tool and a bosh release of the tool that sends Garden debug metrics to the [Wavefront]() board. Used in test deployments to monitor number of go routines and memory used by Garden - https://github.com/masters-of-cats/cfbench - a tool that pushes and scales a test CF app and sends loggregator metrics to the Wavefront dashboard. Used in performance tests on CI - https://github.com/masters-of-cats/linux-playground - an (outdated) Virtualbox VM which we used to implement and contribute fixes to `runc` and `containerd`. See the [Garden Community track document](https://docs.google.com/document/d/1XF2ji6FcDbFCJ6ffkSlTN915yYJKomoGfGF0MmLum-Y/edit) for (possibly outdated) instructions how to use it