-
Notifications
You must be signed in to change notification settings - Fork 33
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #152 from pedro-psb/migrate-to-new-docs-stack
Migrate to new docs stack
- Loading branch information
Showing
28 changed files
with
573 additions
and
1 deletion.
There are no files selected for viewing
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
# Create Custom Profiles | ||
|
||
You can create custom profiles for your instance of oci_env by creating adding `_local` to the end of the profile name. These profiles will be ignored by git. Example `oci_env/profiles/my_custom_profile_local/`. | ||
|
||
For a complete list of available variables, check the [Profile Configuration](site:oci_env/docs/dev/reference/profile-config/) reference. | ||
|
||
## Basic Usage | ||
|
||
OCI env has a pluggable profile system. Profiles can be defined in `oci_env/profiles/` or | ||
in any pulp plugin at `<PLUGIN_NAME>/profiles/`. | ||
|
||
To generate a new profile template run: | ||
|
||
```bash | ||
# Generate a new profile in oci_env | ||
oci-env profile init my_profile | ||
|
||
# Generate a new profile in a plugin repo | ||
oci-env profile init -p PLUGIN_NAME my_profile | ||
|
||
# List available profiles | ||
oci-env profile ls | ||
|
||
# Display the README.md for a profile | ||
oci-env profile docs my_profile | ||
``` | ||
|
||
Each profile goes in it's own directory and can include: | ||
|
||
- `compose.yaml`: This is a docker compose file that can define new services or modify the base `pulp` service. | ||
- `pulp_config.env`: Environment file that defines any settings that the profile needs to run. | ||
- `init.sh`: Script that gets run when the environment loads. Can be used to initialize data and set up tests. Must be a bash script. | ||
- `profile_reqirements.txt`: A list of other profiles that are required to be set in COMPOSE_PROFILE for this profile to function. | ||
- `profile_default_config.env`: A list of default variables to use if not specified by the user. | ||
- `README.md`: Readme file describing what the profile is for and how to use it. | ||
|
||
|
||
## Intertermediate Example | ||
|
||
Profile structure in the galaxy_ng plugin | ||
|
||
``` | ||
galaxy_ng/profiles/ | ||
└── ui | ||
├── compose.yaml | ||
└── pulp_config.env | ||
``` | ||
|
||
compose.yaml: This defines a UI service that builds the container found at `ANSIBLE_HUB_UI_PATH` and configures the UI to proxy requests to the pulp API server. | ||
|
||
```yaml | ||
version: "3.7" | ||
|
||
services: | ||
ui: | ||
build: | ||
context: "{ANSIBLE_HUB_UI_PATH}" | ||
ports: | ||
- "8002:8002" | ||
volumes: | ||
- "{ANSIBLE_HUB_UI_PATH}:/hub/app/" | ||
tmpfs: | ||
# Forces npm to ignore the node_modules in the volume and look | ||
# for it in ../node_modules instead, while still being able to write .cache | ||
- "/hub/app/node_modules" | ||
depends_on: | ||
- pulp | ||
environment: | ||
- "API_PROXY_HOST=pulp" | ||
- "API_PROXY_PORT=80" | ||
- "DEPLOYMENT_MODE=standalone" | ||
``` | ||
pulp_config.env: The UI expects the galaxy apis to be served from `/api/automation-hub/` and for the app to be launched in standalone mode. The environment file provided with the profile ensures that the API is configured correctly to consume the new service. | ||
|
||
``` | ||
PULP_CONTENT_PATH_PREFIX=/api/automation-hub/v3/artifacts/collections/ | ||
|
||
PULP_GALAXY_API_PATH_PREFIX=/api/automation-hub/ | ||
|
||
PULP_GALAXY_COLLECTION_SIGNING_SERVICE=ansible-default | ||
PULP_RH_ENTITLEMENT_REQUIRED=insights | ||
|
||
PULP_ANSIBLE_API_HOSTNAME={API_PROTOCOL}://{API_HOST}:{API_PORT} | ||
PULP_ANSIBLE_CONTENT_HOSTNAME={API_PROTOCOL}://{API_HOST}:{API_PORT}/api/automation-hub/v3/artifacts/collections | ||
|
||
PULP_TOKEN_AUTH_DISABLED=true | ||
|
||
``` | ||
To activate this profile set `COMPOSE_PROFILE=galaxy_ng/ui`. Running this will launch the UI container along with pulp. | ||
```bash | ||
dnewswan-mac:oci_env dnewswan$ docker ps | ||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES | ||
2cc06f79bc87 oci_env_ui "docker-entrypoint.s…" 3 seconds ago Up 1 second 0.0.0.0:8002->8002/tcp oci_env_ui_1 | ||
e1c3ae797018 localhost/oci_env/pulp:base "/init" 6 seconds ago Up 2 seconds 0.0.0.0:5001->80/tcp oci_env_pulp_1 | ||
``` | ||
|
||
Multiple profiles can be selected with `COMPOSE_PROFILE=galaxy_ng/ui:profile2:profile3`. The last profile loaded gets priority on environment variables. Each `compose.yaml` is added additively, and subsquent profles can modify the services from previous profiles. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# Run multiple environments | ||
|
||
You can running multiple environments simultaneously. | ||
|
||
## Create an `.env` file | ||
|
||
You may place it in the root of `oci_env` dir: | ||
|
||
```bash title="custom.env" | ||
|
||
COMPOSE_PROFILE=my_profiles | ||
DEV_SOURCE_PATH=pulpcore | ||
|
||
# These two values must be different from the port and project name for any other instances of the | ||
# environment that are running to avoid conflicts. | ||
API_PORT=4002 | ||
COMPOSE_PROJECT_NAME=test | ||
|
||
# If you want to use a different directory for your git checkouts you can set this | ||
# SRC_DIR=/path/to/my/git/checkouts | ||
``` | ||
|
||
## Launch the Environment | ||
|
||
If you are in the same dir as the file, you can run: | ||
|
||
```bash | ||
oci-env -e custom.env compose up | ||
``` | ||
|
||
## Run from anywhere | ||
|
||
If you have `OCI_ENV_PATH` defined you can create a directory for your custom definitions and run oci-env from there without having to specify an absolute path. Example: | ||
|
||
```bash | ||
$ tree | ||
~ | ||
├── oci_env | ||
└── oci_env_configs | ||
├── custom.env | ||
└── test.env | ||
$ export OCI_ENV_PATH="~/oci_env" | ||
$ cd oci_env_configs | ||
$ oci-env -e custom.env compose up | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
# Run Tests | ||
|
||
## Lint | ||
|
||
```bash | ||
# Install the lint requirements and run the linter for a specific plugin | ||
oci-env test -i -p PLUGIN_NAME lint | ||
|
||
# Run the linter without installing lint dependencies. | ||
oci-env test -p PLUGIN_NAME lint | ||
``` | ||
|
||
## Unit | ||
|
||
```bash | ||
# Install the unit test dependencies for a plugin and run it. | ||
oci-env test -i -p PLUGIN_NAME unit | ||
|
||
# Run the unit tests for a plugin without installing test dependencies. | ||
oci-env test -p PLUGIN_NAME unit | ||
``` | ||
|
||
## Functional | ||
|
||
Before functional tests can be run, you must clone `github.com/pulp/pulp-openapi-generator` into the parent directory: | ||
|
||
``` | ||
. | ||
├── (...) | ||
├── oci_env | ||
└── pulp-openapi-generator | ||
``` | ||
|
||
```bash | ||
# Generate the pulp client. This will build clients for all plugins in DEV_SOURCE_PATH. -i will also install the client in the container. | ||
oci-env generate-client -i | ||
|
||
# Install the functional test requirements and run the tests | ||
oci-env test -i -p PLUGIN_NAME functional | ||
|
||
# Run the tests without installing dependencies. | ||
oci-env test -p PLUGIN_NAME functional | ||
``` | ||
|
||
Bindings for specific plugins can be regenerated with `oci-env generate-client PLUGIN_NAME`. | ||
|
||
Use `--language` parameter to specify the programming language the bindings should be generated for (default: python), | ||
e.g. `oci-env generate-client -l ruby PLUGIN_NAME`. | ||
|
||
## Remote Debugging | ||
|
||
### Using `epdb` | ||
|
||
1. Add "epdb" to the `functest_requirements.txt` file in your pulp_ansible checkout path. | ||
2. Inside any functional test, add `import epdb; epdb.st()`. | ||
3. Re-run `oci-env test -i functional` and `oci-env test -p pulp_ansible functional --capture=no` commands again. | ||
|
||
### Using PyCharm | ||
|
||
1. Start the debugger server in PyCharm. When using `podman`, the hostname should be set to | ||
`host.containers.internal` hostname. `Docker` users should use `host.docker.internal` hostname. | ||
2. Add a break point to your Pythong code: | ||
``` | ||
import pydevd_pycharm | ||
pydevd_pycharm.settrace('host.containers.internal', port=3013, stdoutToServer=True, stderrToServer=True)` | ||
``` | ||
3. Restart all services you need to pick up the code change by running `s6-svc -r /var/run/service/<service_name>` | ||
4. Perform the action that should trigger the code to run. | ||
|
||
### Caveat | ||
|
||
Please note that `host.containers.internal` points to the wrong interface in `podman < 4.1`. When | ||
using `podman < 4.1`, you need modify `/etc/hosts` inside the container running Pulp with the IP | ||
address for the publicly facing network interface on the host. | ||
|
2 changes: 1 addition & 1 deletion
2
docs/macos_troubleshooting_tips.md → ...bleshooting/macos_troubleshooting_tips.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
# macOS troubleshooting tips | ||
# MacOS machines | ||
|
||
## Setup on arm64 macOS machines | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# OCI Env | ||
|
||
OCI Env is a developer environment for pulp based off of the [Pulp OCI Images](https://github.com/pulp/pulp-oci-images). | ||
|
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# How it Works | ||
|
||
At it's core, the oci-env command launches a predictable set of containers with a CLI interface to communicate with them. Thes containers are launched by taking advantage of a feature in docker and podman compose that allow multiple compose.yaml files to be selected via the `-f` flag. The command that `oci-env` runs can be viewed with the `-v` flag: | ||
|
||
```bash | ||
oci-env -v compose up | ||
Running command in container: docker-compose -p oci_env -f /Users/dnewswan/code/hub/oci_env/.compiled/oci_env/base_compose.yaml -f /Users/dnewswan/code/hub/oci_env/.compiled/oci_env/galaxy_ui_compose.yaml up | ||
``` | ||
|
||
Since not all compose runtimes support variable interpolation, oci-env handles that by itself. The compose.yaml files provided by all of the plugins are gathered up and the variables defined in your `compose.env` file are substituted using python's `str.format()` command and placed in the `.compiled/<COMPOSE_PROJECT_NAME>/` directory. This directory contains all the information for the running instance of your dev enviornment: | ||
|
||
```bash | ||
(venv) dnewswan-mac:oci_env dnewswan$ tree .compiled/ | ||
.compiled/ | ||
├── ci | ||
│ ├── base_compose.yaml | ||
│ ├── combined.env | ||
│ └── init.sh | ||
└── oci_env | ||
├── base_compose.yaml | ||
├── combined.env | ||
├── galaxy_ui_compose.yaml | ||
└── init.sh | ||
``` | ||
|
||
`.compiled/` contains all your compose files (with the correct variable substitutions) as well as an `init.sh` script that launches each profile's init.sh script and a combined.env file which combines all the pulp_config.env files into one and performs variable substitution. The `combined.env` file is then loaded into the pulp container as an environment variable, and `init.sh` is run once the container has initialized. | ||
|
||
Once all of the information here is compiled, `oci-env` launches the container runtime and mounts the following directories: | ||
|
||
- oci_env is mounted into `/opt/oci_env/`. This creates a predictable location to launch scripts provided by oci_env (such as `/opt/oci_env/base/container_scripts/run_functional_tests.sh`) | ||
- your source code directory is mountent into `/opt/src/`. This provies a predictable location to find plugin source code (such as `/src/pulpcore/`). |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Profile Configuration | ||
|
||
For creating custom profiles, see [Create Custom Profiles](site:oci_env/docs/dev/guides/create-custom-profiles/). | ||
|
||
## Available variables | ||
These variables can be used in `pulp_config.env` and `compose.yaml`: | ||
|
||
- `API_HOST`: hostname where pulp expects to run (default: localhost). | ||
- `API_PORT`: port that pulp expects to run on. This port will also get exposed on the pulp container (default: 5001). | ||
- `API_PROTOCOL`: can be http or https (default: http). | ||
- `NGINX_SSL_PORT`: the port on which Nginx listens to https traffic (default: 443). `API_PROTOCOL` needs to be `https`. | ||
- `NGINX_PORT`: the port on which Nginx listens to http traffic. Note: the functional tests won't work correctly if this is different from API_PORT. (default: 5001, or the value of `API_PORT`). | ||
- `DEV_SOURCE_PATH`: colon separated list of python dependencies to include from source. | ||
- `COMPOSE_PROFILE`: colon separated list of profiles. | ||
- `DJANGO_SUPERUSER_USERNAME`: username for the super user (default: admin). | ||
- `DJANGO_SUPERUSER_PASSWORD`: password for the super user (default: password). | ||
- `COMPOSE_PROJECT_NAME`: the project name passed to podman-compose. Use this when running multiple environments to keep containers and volumes separate (default: oci_env). | ||
- `SRC_DIR`: path to load source code from. Set this if you want to use a different set of git checkouts with your environment (default: oci_env/../) | ||
|
||
## Notes | ||
|
||
- Variables are templated using pythons `"{VAR}".template(VAR="my_var")` function, so they must be referenced as `{VARIABLE_NAME}` in environment and compose files. | ||
- Profiles can use variables outside of this list as well. They are just required to be defined in the user's compose.env file or in the profile's (or its parents) `profile_default_config.env` file. | ||
|
||
Example: | ||
|
||
```bash title="pulp_config.env" | ||
PULP_ANSIBLE_API_HOSTNAME="{API_PROTOCOL}://{API_HOST}:{API_PORT}" | ||
PULP_ANSIBLE_CONTENT_HOSTNAME="{API_PROTOCOL}://{API_HOST}:{API_PORT}/pulp/content" | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# galaxy_base | ||
|
||
## Usage | ||
|
||
*Add a description detailing what this profile does and any extra instructions for using it.* | ||
|
||
This provides the basic settings required to run Galaxy NG. | ||
|
||
## Extra Variables | ||
|
||
*List any extra variables that user's can configure in their .compose.env* | ||
|
||
- `ENABLE_SIGNING` | ||
- Description: Enable or disable the signing service | ||
- Options: | ||
- 0: disable signing | ||
- 1: setup keyrings and create default signing service | ||
- 2: setup keyrings, but don't create a signing service | ||
- Default: 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# galaxy_ui | ||
|
||
## Usage | ||
|
||
*Add a description detailing what this profile does and any extra instructions for using it.* | ||
|
||
Provides a container that runs the Galaxy NG UI from source. This requires https://github.com/ansible/ansible-hub-ui to be checked out on your system. | ||
|
||
## Extra Variables | ||
|
||
*List any extra variables that user's can configure in their .compose.env* | ||
|
||
- `ANSIBLE_HUB_UI_PATH` | ||
- Description: The location where ansible-hub-ui is checked out on your system. | ||
- Default: Unset. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# local\_fixtures | ||
|
||
A profile for running test fixtures locally. It pulls down the pulp/pulp-fixtures image which | ||
can be used for running functional tests without relying on the internet connection. The fixtures | ||
are exposed on the very same port on the host machine. | ||
|
||
## Usage | ||
|
||
Append `local_fixtures` to the `COMPOSE_PROFILE` variable in your `compose.env`. | ||
|
||
## Extra Variables | ||
|
||
- `PULP_FIXTURES_URL` | ||
- Description: The URL/origin for the test fixtures endpoint. | ||
- Default: "http://local\_fixtures:8080" |
Oops, something went wrong.