Skip to content

Commit

Permalink
chore: apl-core transfer (#1639)
Browse files Browse the repository at this point in the history
  • Loading branch information
j-zimnowoda committed Jul 8, 2024
1 parent bfe3510 commit 22eef0f
Show file tree
Hide file tree
Showing 24 changed files with 82 additions and 85 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ on:
default: 'yes'
env:
CACHE_REGISTRY: ghcr.io
CACHE_REPO: redkubes/otomi-core
CACHE_REPO: linode/apl-core
REPO: otomi/core
GIT_USER: svcAPLBot
SCALEWAY_NODE_TYPE: PRO2-M
Expand Down
45 changes: 21 additions & 24 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,43 +7,36 @@ on:
- '*'
env:
CACHE_REGISTRY: ghcr.io
CACHE_REPO: redkubes/otomi-core
CACHE_REPO: linode/apl-core
REPO: otomi/core
GIT_USER: redkubesbot

jobs:
build-test-cache:
if: (!contains(github.event.head_commit.message, 'ci skip') && !startsWith(github.ref, 'refs/tags/') && !github.event.act)
runs-on: ubuntu-latest
env:
COMMIT_MSG: ${{ github.event.head_commit.message }}
BOT_EMAIL: ${{ vars.BOT_EMAIL }}
BOT_USERNAME: ${{ vars.BOT_USERNAME }}
steps:
- name: Set env
run: |
tag=${GITHUB_REF##*/}
echo "Creating tag: $tag"
echo "TAG=$tag" >> $GITHUB_ENV
git config --global user.email $GIT_USER@users.noreply.github.com
git config --global user.name $GIT_USER
git config --global user.email $BOT_EMAIL
git config --global user.name $BOT_USERNAME
- name: Checkout
uses: actions/checkout@v4
# - name: CI tests, image build and push tag for main or branch
# uses: whoan/docker-build-with-cache-action@v5
# with:
# username: ${{ env.GIT_USER }}
# password: '${{ secrets.NPM_TOKEN }}'
# registry: ${{ env.CACHE_REGISTRY }}
# image_name: ${{ env.CACHE_REPO }}
# image_tag: ${{ env.TAG }}
# pull_image_and_stages: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Github Packages
uses: docker/login-action@v3
with:
registry: ${{ env.CACHE_REGISTRY }}
username: ${{ env.GIT_USER }}
password: '${{ secrets.NPM_TOKEN }}'
username: ${{ env.BOT_USERNAME }}
password: '${{ secrets.BOT_TOKEN }}'
- name: CI tests, image build and push tag for main or branch
uses: docker/build-push-action@v5
with:
Expand All @@ -55,11 +48,15 @@ jobs:
needs: build-test-cache
if: always() && ((contains(needs.build-test-cache.result, 'success') && !contains(needs.integration.outputs.started, 'true')) || (contains(needs.integration.result, 'success'))) && !github.event.act
runs-on: ubuntu-latest
env:
BOT_EMAIL: ${{ vars.BOT_EMAIL }}
BOT_USERNAME: ${{ vars.BOT_USERNAME }}
steps:
- name: Push to docker hub
run: |
set -u
TAG=${GITHUB_REF##*/}
docker login -u $GIT_USER -p '${{ secrets.NPM_TOKEN }}' ghcr.io
docker login ghcr.io -u $BOT_USERNAME -p ${{ secrets.BOT_TOKEN }}
image="$CACHE_REGISTRY/$CACHE_REPO:$TAG"
docker pull $image
docker tag $image $REPO:$TAG
Expand All @@ -80,8 +77,8 @@ jobs:
uses: actions/checkout@v4
- name: Set env
run: |
git config --global user.email $GIT_USER@users.noreply.github.com
git config --global user.name $GIT_USER
git config --global user.email $BOT_EMAIL
git config --global user.name $BOT_USERNAME
- name: Create and push git tag
id: git_tag
run: |
Expand All @@ -95,10 +92,10 @@ jobs:
echo "Releasing $REPO:$release_tag"
docker tag $REPO:$TAG $REPO:$release_tag
docker push $REPO:$release_tag
docker login -u $GIT_USER -p '${{ secrets.NPM_TOKEN }}' ghcr.io
docker login -u $BOT_USERNAME -p '${{ secrets.BOT_TOKEN }}' ghcr.io
docker tag $REPO:$TAG $CACHE_REGISTRY/$CACHE_REPO:$release_tag
docker push $CACHE_REGISTRY/$CACHE_REPO:$release_tag
echo "machine github.com login $GIT_USER password ${{ secrets.GIT_PASSWORD }}" > ~/.netrc
echo "machine github.com login ${{ env.BOT_USERNAME }} password ${{ secrets.BOT_TOKEN }}" > ~/.netrc
git tag -am "$COMMIT_MSG" $release_tag && git push --follow-tags
#Cut the CHANGELOG.md file up to the first occurence of the "### \[[0-9]*" (meaning three #, a space,a square bracket and any number after it)
sed -n '/### \[[0-9]*/q;p' CHANGELOG.md > NEW_CHANGELOG.md
Expand All @@ -125,7 +122,7 @@ jobs:
id: prepare_chart
run: |
# Install and update helm repo
helm repo add otomi https://otomi.io/otomi-core
helm repo add otomi https://otomi.io/apl-core
helm repo update
# Retrieve the app version from package.json
Expand Down Expand Up @@ -164,8 +161,8 @@ jobs:
npx js-yaml values-schema.yaml > chart/otomi/values.schema.json
# Set the global id for git as it seems needed by the next step when a custom image is used
git config --global user.email $GIT_USER@users.noreply.github.com
git config --global user.name $GIT_USER
git config --global user.email ${{ env.BOT_EMAIL }}
git config --global user.name ${{ env.BOT_USERNAME }}
- name: Create and publish otomi chart release
id: chart_release
uses: helm/chart-releaser-action@v1.6.0
Expand Down Expand Up @@ -194,7 +191,7 @@ jobs:
- name: Pulling the helm chart
run: |
# Install and update helm repo
helm repo add otomi https://otomi.io/otomi-core
helm repo add otomi https://otomi.io/apl-core
helm repo update
# Get latest version of otomi
Expand Down
2 changes: 1 addition & 1 deletion .values/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

Part of Otomi Container Platform.

Repo that holds cluster configuration to be used by [otomi-core](https://github.com/redkubes/otomi-core).
Repo that holds cluster configuration to be used by [apl-core](https://github.com/linode/apl-core).

Please read the online [otomi documentation](https://otomi.io/) for instructions on how to work with the platform.
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@ Otomi has been acquired by <a href="https://www.linode.com/otomi/">Akamai</a>. W
</h3>

<p align="center">
<a href="https://github.com/redkubes/otomi-core/releases/"><img alt="Releases" src="https://img.shields.io/github/release-date/redkubes/otomi-core?label=latest%20release" /></a>
<a href="https://img.shields.io/github//redkubes/otomi-core/actions/workflows/main.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/redkubes/otomi-core/main.yml" /></a>
<a href="https://img.shields.io/github/last-commit/redkubes/otomi-core"><img alt="Last commit" src="https://img.shields.io/github/last-commit/redkubes/otomi-core" /></a>
<a href="https://github.com/linode/apl-core/releases/"><img alt="Releases" src="https://img.shields.io/github/release-date/linode/apl-core?label=latest%20release" /></a>
<a href="https://img.shields.io/github//linode/apl-core/actions/workflows/main.yml"><img alt="Build status" src="https://img.shields.io/github/actions/workflow/status/linode/apl-core/main.yml" /></a>
<a href="https://img.shields.io/github/last-commit/linode/apl-core"><img alt="Last commit" src="https://img.shields.io/github/last-commit/linode/apl-core" /></a>
<a href="https://img.shields.io/crates/l/ap"><img alt="License" src="https://img.shields.io/crates/l/ap" /></a>
<a href="https://img.shields.io/badge/contributions-welcome-orange.svg"><img alt="Contributions" src="https://img.shields.io/badge/contributions-welcome-orange.svg" /></a>
<a href="http://otomi.io/"><img src="https://img.shields.io/website-up-down-green-red/http/shields.io.svg" alt="Website otomi.io"></a>
<a href="https://join.slack.com/t/otomi/shared_invite/zt-1axa4vima-E~LHN36nbLR~ay5r5pGq9A"><img src="https://img.shields.io/badge/slack--channel-blue?logo=slack"></a>
</p>

<p align="center"><img src="https://github.com/redkubes/otomi-core/blob/main/docs/img/otomi-console.png/?raw=true" width="100%" align="center" alt="Otomi integrated applications"></p>
<p align="center"><img src="https://github.com/linode/apl-core/blob/main/docs/img/otomi-console.png/?raw=true" width="100%" align="center" alt="Otomi integrated applications"></p>

<h4 align="center">
Otomi turns any Kubernetes cluster into an Application Platform to provide paved roads from code to production
Expand All @@ -42,7 +42,7 @@ To install Otomi, make sure to have a K8s cluster running with at least:
Add the Helm repository:

```bash
helm repo add otomi https://otomi.io/otomi-core
helm repo add otomi https://otomi.io/apl-core
helm repo update
```

Expand Down Expand Up @@ -147,15 +147,15 @@ If you wish to contribute please read our [Contributor Code of Conduct](https://

If you want to say **thank you** or/and support the active development of Otomi:

- [Star](https://github.com/redkubes/otomi-core) the Otomi project on Github
- [Star](https://github.com/linode/apl-core) the Otomi project on Github
- Feel free to write articles about the project on [dev.to](https://dev.to/), [medium](https://medium.com/) or on your personal blog and share your experiences

This project exists thanks to all the people who have contributed

<a href="https://github.com/redkubes/otomi-core/graphs/contributors">
<img src="https://contrib.rocks/image?repo=redkubes/otomi-core" />
<a href="https://github.com/linode/apl-core/graphs/contributors">
<img src="https://contrib.rocks/image?repo=linode/apl-core" />
</a>

## License

Otomi is licensed under the [Apache 2.0 License](https://github.com/redkubes/otomi-core/blob/main/LICENSE).
Otomi is licensed under the [Apache 2.0 License](https://github.com/linode/apl-core/blob/main/LICENSE).
28 changes: 14 additions & 14 deletions adr/index.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# Architectural Decision Log

This log lists the architectural decisions for otomi-core.
This log lists the architectural decisions for apl-core.

<!-- adrlog -- Regenerate the content by using "npm run adr". -->

* [ADR-2020-10-01](2020-10-01-github-workflow.md) - Our GitHub workflow
* [ADR-2020-10-02](2020-10-02-docker-compose.md) - docker compose
* [ADR-2020-11-06](2020-11-06-bash-style-guide.md) - Bash coding standard
* [ADR-2020-11-06](2020-11-06-konstraint-policy-library.md) - Konstraint library to support Common Rego Language definitions
* [ADR-2021-10-18](2021-10-18-defaults-and-derived.md) - Derived values
* [ADR-2021-10-28](2021-10-28-internal-values.md) - Internal values
* [ADR-2022-03-24](2022-03-24-custom-ca.md) - urice:
* [ADR-2022-04-22](2022-04-22-values-migration.md) - Values migration
* [ADR-2022-04-23](2022-04-23-pre-upgrade.md) - A new otomi pre-upgrade command
* [ADR-2022-05-17](2022-05-17-destroy-upon-uninstall.md) - Extra flags to accomodate destroy upon uninstall
* [ADR-2022-06-07](2022-06-07-ingress-classes.md) - Ingress classes
* [ADR-2022-07-02](2022-07-02-node-affinity.md) - Node affinity
* [ADR-2022-08-26](2022-08-26-other-dns-provider.md) - Other DNS provider
- [ADR-2020-10-01](2020-10-01-github-workflow.md) - Our GitHub workflow
- [ADR-2020-10-02](2020-10-02-docker-compose.md) - docker compose
- [ADR-2020-11-06](2020-11-06-bash-style-guide.md) - Bash coding standard
- [ADR-2020-11-06](2020-11-06-konstraint-policy-library.md) - Konstraint library to support Common Rego Language definitions
- [ADR-2021-10-18](2021-10-18-defaults-and-derived.md) - Derived values
- [ADR-2021-10-28](2021-10-28-internal-values.md) - Internal values
- [ADR-2022-03-24](2022-03-24-custom-ca.md) - urice:
- [ADR-2022-04-22](2022-04-22-values-migration.md) - Values migration
- [ADR-2022-04-23](2022-04-23-pre-upgrade.md) - A new otomi pre-upgrade command
- [ADR-2022-05-17](2022-05-17-destroy-upon-uninstall.md) - Extra flags to accomodate destroy upon uninstall
- [ADR-2022-06-07](2022-06-07-ingress-classes.md) - Ingress classes
- [ADR-2022-07-02](2022-07-02-node-affinity.md) - Node affinity
- [ADR-2022-08-26](2022-08-26-other-dns-provider.md) - Other DNS provider

<!-- adrlogstop -->

Expand Down
2 changes: 1 addition & 1 deletion binzx/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# How to use the new otomi cli

```sh
# In the otomi-core directory
# In the apl-core directory
export DOCKER_TAG=binzx
docker build --target prod -t otomi/core:binzx .
./binzx/otomi <commands here>
Expand Down
2 changes: 1 addition & 1 deletion binzx/installer
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
readonly script=otomi
readonly current_branch=$(git rev-parse --abbrev-ref HEAD)
readonly branch=${OTOMI_TAG:-$current_branch}
readonly remote=https://raw.githubusercontent.com/redkubes/otomi-core/$branch/binzx/$script
readonly remote=https://raw.githubusercontent.com/linode/apl-core/$branch/binzx/$script
readonly path=/usr/local/bin

readonly msg="Please choose the default install location [$path]: "
Expand Down
10 changes: 5 additions & 5 deletions binzx/otomi
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
#####################################################################################
##
## NOTE:
## This is a command line tool to operate on otomi-core.
## This is a command line tool to operate on apl-core.
## All commands are executed in docker container.
## Keep this file as simple as possible:
## - do not depend on any external files.
## - do not use any non standard tooling.
## - only Docker is needed to run otomi-core image
## - only Docker is needed to run apl-core image
## If you need to use any extra binaries then most probably you want to add them to the otomi/tools image.
##
#####################################################################################
Expand Down Expand Up @@ -97,7 +97,7 @@ if [[ ${BASH_SOURCE[0]} == '/'* ]]; then
script_full_path="${BASH_SOURCE[0]}"
fi
readonly script_full_path
update_base="https://raw.githubusercontent.com/redkubes/otomi-core/${otomi_branch_or_tag}/binzx/otomi"
update_base="https://raw.githubusercontent.com/linode/apl-core/${otomi_branch_or_tag}/binzx/otomi"

function date_to_format() {
date=${1:-'1970-01-01T00:00:00Z'}
Expand Down Expand Up @@ -164,7 +164,7 @@ check_update() {
[[ $calling_args == *'-ni'* ]]; then
return 0
fi
last_commit_date=$(curl -s "https://api.github.com/repos/redkubes/otomi-core/commits?sha=${otomi_branch_or_tag}&path=binzx%2Fotomi&page=1&per_page=1" | grep -A5 '"committer":' | grep '"date":' | awk -F': ' '{print $2}' | tr -d '"')
last_commit_date=$(curl -s "https://api.github.com/repos/linode/apl-core/commits?sha=${otomi_branch_or_tag}&path=binzx%2Fotomi&page=1&per_page=1" | grep -A5 '"committer":' | grep '"date":' | awk -F': ' '{print $2}' | tr -d '"')
last_file_change=$(date -u -r $script_full_path '+%Y-%m-%dT%H:%M:%SZ')
last_commit_date_sec=$(date_to_format $last_commit_date)
last_file_change_sec=$(date_to_format $last_file_change)
Expand Down Expand Up @@ -307,7 +307,7 @@ fi

mkdir -p /tmp/otomi
# Issue arises due to OSX not assuming UTF-8 format: https://unix.stackexchange.com/a/64905 need to set LC_ALL=C
container_name="otomi-core-$(cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 6 | head -n 1)"
container_name="apl-core-$(cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 6 | head -n 1)"

if [ "${IN_DOCKER,,}" == "false" ]; then
silent echo $cmd
Expand Down
2 changes: 1 addition & 1 deletion chart/otomi/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ keywords:
- metrics
- tracing
sources:
- https://github.com/redkubes/otomi-core
- https://github.com/linode/apl-core
10 changes: 5 additions & 5 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Effective development starts with an understanding the code structure and the re
## Project structure

```
otomi-core
apl-core
├── .values # Boilerplate for initializing git repository
├── adr # Architectural Decision Records [read more](https://adr.github.io/madr/)
├── bin # Otomi CLI entrypoint (deprecated)
Expand All @@ -44,7 +44,7 @@ otomi-core
It is important that you get familiar with reusable code snippets located at `helmfile.d/snippets`. Below, I describe the most essential ones:

```
otomi-core/helmfile.d/snippets
apl-core/helmfile.d/snippets
├── defaults.yaml # static defaults that can be overwritten by user values and/or derived values
├── derived.gotmpl # values derived from default and user values
├── env.gotmpl # define helmfile environment settings
Expand Down Expand Up @@ -141,7 +141,7 @@ In the [Code structure](#Code-structure) chapter, you have learned about reusabl

> We present `myapp` integration. The `myapp` is an arbitrary name and can be anything else.
> A core apps are those defined in otomi-core. Do not confuse them with team services, which are defined in the values repo.
> A core apps are those defined in apl-core. Do not confuse them with team services, which are defined in the values repo.
## Using the default code snippet

Expand Down Expand Up @@ -316,11 +316,11 @@ TBD

## Using CLI while developing templates

Using Otomi CLI can be very helpful while integrating apps or developing new features that involve the execution of Helmfile because it allows you to render and validate manifests. It is possible to use Otomi CLI in development mode, so the Otomi CLI reflects changes made in your local `otomi-core` directory.
Using Otomi CLI can be very helpful while integrating apps or developing new features that involve the execution of Helmfile because it allows you to render and validate manifests. It is possible to use Otomi CLI in development mode, so the Otomi CLI reflects changes made in your local `apl-core` directory.

To run Otomi CLI in the development mode, you must:

- execute Otomi CLI commands from a root directory of the `otomi-core` project
- execute Otomi CLI commands from a root directory of the `apl-core` project
- export `ENV_DIR`

First, run `npm install` to build all modules required for CLI.
Expand Down
8 changes: 4 additions & 4 deletions docs/migrating-values.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Assumptions/conventions used in this document:

- `otomi-values` is a git repo with values bootstrapped by a version of an `otomi-core` container.
- `otomi-values` is a git repo with values bootstrapped by a version of an `apl-core` container.

### The problem

Expand All @@ -13,11 +13,11 @@ We break the schema (`values-schema.yaml`) in our `otomi-values` when:
- We move props around
- We mutate a property's type and/or it's value shape

In order to move to another version of `otomi-core` (which might contain some of these changes), we needed a mechanism that migrates the old `otomi-values` to conform to the new schema. So we came up with a simple approach that serves our simple schema.
In order to move to another version of `apl-core` (which might contain some of these changes), we needed a mechanism that migrates the old `otomi-values` to conform to the new schema. So we came up with a simple approach that serves our simple schema.

### Solution

By comparing the diff between the old and the new schema a developer is able to create change records in `otomi-core` and store them in `values-changes.yaml`. Each record is given a successive number. If a user wants to upgrade the platform to a newer version of `otomi-core`, these change records will have to be applied to the values in successive order. When a dev runs `otomi commit` in an `otomi-values` repo, changes made to `otomi.version` will be detected, and the migration script will run in the newer container. The script only needs to compare the new `version` in `values-schema.yaml` with the `version` found in `otomi-values`.
By comparing the diff between the old and the new schema a developer is able to create change records in `apl-core` and store them in `values-changes.yaml`. Each record is given a successive number. If a user wants to upgrade the platform to a newer version of `apl-core`, these change records will have to be applied to the values in successive order. When a dev runs `otomi commit` in an `otomi-values` repo, changes made to `otomi.version` will be detected, and the migration script will run in the newer container. The script only needs to compare the new `version` in `values-schema.yaml` with the `version` found in `otomi-values`.

### Change records

Expand Down Expand Up @@ -72,7 +72,7 @@ changes:

### Workflow

Every time a developer changes the schema (`values-schema.yaml`) and adds change records, the `otomi migrate` command should be performed that will migrate the values to the new structure. For the following example workflow, we assume that the current (new) schema version of `otomi-core` is `4`, and the previous `otomi-values` version is at `3`.
Every time a developer changes the schema (`values-schema.yaml`) and adds change records, the `otomi migrate` command should be performed that will migrate the values to the new structure. For the following example workflow, we assume that the current (new) schema version of `apl-core` is `4`, and the previous `otomi-values` version is at `3`.

Note:

Expand Down
Loading

0 comments on commit 22eef0f

Please sign in to comment.