Recode

Remote development environments defined as code. Running in your cloud provider account.
Currently available on Amazon Web Services and Visual Studio Code.

... you can think of it as a desktop version of Gitpod / Coder / GitHub Codespaces less polished and with less features but 100% free, 100% open-source and 100% community-driven

recode aws start recode-sh/workspace --instance-type t2.medium

recode_intro.mp4

... see the recode-sh/workspace repository for an example of development environment configuration

Table of contents

• Requirements
• Installation
• Usage
  • Login
  • Start
  • Stop
  • Remove
  • Uninstall
• Development environments configuration
  • Tip: the --rebuild flag
  • Tip: Docker & Docker compose
  • User configuration
  • Project configuration
    • What if I don't have an user configuration?
  • Recode configuration
    • Base image (recode-sh/base-dev-env)
    • Visual Studio Code extensions
    • Multiple repositories
    • Build arguments (RECODE_INSTANCE_OS and RECODE_INSTANCE_ARCH)
    • Hooks
• Frequently asked questions
  • How does it compare with GitPod / Coder / Codespaces / X?
  • How does it compare with Vagrant / VSCode remote SSH / Container extensions?
  • Why using Docker as a VM and not something like NixOS, for example?
  • Given that my dev env will run in a container does it mean that it will be limited?
• The future
• License

Requirements

The Recode binary has been tested on Linux and Mac. Support for Windows is theoretical (testers needed 💙).

Before using Recode, the following dependencies need to be installed:

• Visual Studio Code (currently the sole editor supported).

• OpenSSH Client (used to access your development environments).

Installation

The easiest way to install Recode is by running the following command in your terminal:

curl -sf https://raw.githubusercontent.com/recode-sh/cli/main/install.sh | sh -s -- -b /usr/local/bin latest

This command could be run as-is or by changing:

• The installation directory by replacing /usr/local/bin with your preferred path.

• The version installed by replacing latest with a specific version.

Once done, you could confirm that Recode is installed by running the recode command:

recode --help

Usage

To begin, run the command "recode login" to connect your GitHub account.	

From there, the most common workflow is:

  - recode <cloud_provider> start <repository>  : to start a development environment for a specific GitHub repository
  - recode <cloud_provider> stop <repository>   : to stop a development environment (without removing your data)
  - recode <cloud_provider> remove <repository> : to remove a development environment AND your data
  
<repository> may be relative to your personal GitHub account (eg: cli) or fully qualified (eg: my-organization/api).

Usage:
  recode [command]

Available Commands:
  aws         Use Recode on Amazon Web Services
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  login       Connect a GitHub account to use with Recode

Flags:
  -h, --help    help for recode
  -v, --version version for recode

Use "recode [command] --help" for more information about a command.

Login

recode login

To begin, you need to run the login command to connect your GitHub account.

Recode requires the following permissions:

• "Public SSH keys" and "Repositories" to let you access your repositories from your development environments.

• "GPG Keys" and "Personal user data" to configure Git and sign your commits (verified badge).

All your data (including the OAuth access token) are only stored locally in ~/.config/recode/recode.yml (or in XDG_CONFIG_HOME if set).

The source code that implements the GitHub OAuth flow is located in the recode-sh/api repository.

Start

recode <cloud_provider> start <repository>

The start command creates and starts a development environment for a specific GitHub repository.

If a development environment is stopped, it will only be started. If a development environment is already started, only your code editor will be opened.

An --instance-type flag could be passed to specify the instance type that will power your development environment. (See the corresponding cloud provider repository for default / valid values).

Examples

recode aws start recode-sh/workspace

recode aws start recode-sh/workspace --instance-type t2.medium

Stop

recode <cloud_provider> stop <repository>

The stop command stops a started development environment.

Stopping means that the underlying instance will be stopped but your data will be conserved. You may want to use this command to save costs when the development environment is not used.

Example

recode aws stop recode-sh/workspace

Remove

recode <cloud_provider> remove <repository>

The remove command removes an existing development environment.

Removing means that the underlying instance and all your data will be permanently removed.

Example

recode aws remove recode-sh/workspace

Uninstall

recode <cloud_provider> uninstall

The uninstall command removes all the infrastructure components used by Recode from your cloud provider account. (See the corresponding cloud provider repository for details).

Before running this command, all development environments need to be removed.

Example

recode aws uninstall

Development environments configuration

If you think about all the projects you've worked on, you may notice that you've:

• a set of configuration / tools used for all your projects (eg: a preferred timezone / locale, a specific shell...);

• a set of configuration / tools specific for each project (eg: docker compose, go >= 1.18 or node.js >= 14).

This is what Recode has tried to mimic with user and project configuration.

💡 Tip: the --rebuild flag

recode aws start recode-sh/workspace --rebuild

If you update the configuration of an existing development environment, you could use the --rebuild flag of the start command to rebuild it without having to delete it first.

💡 Tip: Docker & Docker compose

Docker and Docker compose are already preinstalled in all development environments so you don't have to install them.

User configuration

User configuration corresponds to the set of configuration / tools used for all your projects. To create an user configuration, all you need to do is to:

1. Create a repository named .recode in your personal GitHub account.

2. Add a file named dev_env.Dockerfile in it.

The file dev_env.Dockerfile is a regular Dockerfile except that:

• it must derive from recode-sh/base-dev-env (more below);

• the user configuration needs to be applied to the user recode.

Otherwise, you are free to do what you want with this file and this repository. You could see an example with dotfiles in recode-sh/.recode and use it as a GitHub repository template:

# User's dev env image must derive from recodesh/base-dev-env.
# See https://github.com/recode-sh/base-dev-env/blob/main/Dockerfile for source.
FROM recodesh/base-dev-env:latest

# Set timezone
ENV TZ=America/Los_Angeles

# Set locale
RUN sudo locale-gen en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en 
ENV LC_ALL=en_US.UTF-8

# Install Zsh
RUN set -euo pipefail \
  && sudo apt-get --assume-yes --quiet --quiet update \
  && sudo apt-get --assume-yes --quiet --quiet install zsh \
  && sudo rm --recursive --force /var/lib/apt/lists/*

# Install OhMyZSH and some plugins
RUN set -euo pipefail \
  && sh -c "$(curl --fail --silent --show-error --location https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" \
  && git clone --quiet https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions \
  && git clone --quiet https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting

# Change default shell for user "recode"
RUN set -euo pipefail \
  && sudo usermod --shell $(which zsh) recode

# Add all dotfiles to home folder
COPY --chown=recode:recode ./dotfiles/.* $HOME/

Project configuration

Project configuration corresponds to the set of configuration / tools specific for each project. As you may have guessed, to create a project configuration, all you need to do is to:

1. Create a directory named .recode in your project's repository.

2. Add a file named dev_env.Dockerfile in it.

The file dev_env.Dockerfile is a regular Dockerfile except that:

• it must derive from user_dev_env (your user configuration);

• the user configuration needs to be applied to the user recode.

Otherwise, you are free to do what you want with this file and this directory. You could see an example in recode-sh/workspace:

# Project's dev env image must derive from "user_dev_env"
# (ie: github_user_name/.recode/dev_env.Dockerfile)
FROM user_dev_env

# VSCode extensions that need to be installed (optional)
LABEL sh.recode.vscode.extensions="golang.go, zxh404.vscode-proto3, ms-azuretools.vscode-docker"

# GitHub repositories that need to be cloned (optional) (default to the current one)
LABEL sh.recode.repositories="cli, agent, recode, aws-cloud-provider, base-dev-env, api, .recode, workspace"

# Reserved args (RECODE_*). Provided by Recode.
# eg: linux
ARG RECODE_INSTANCE_OS
# eg: amd64 or arm64
ARG RECODE_INSTANCE_ARCH

ARG GO_VERSION=1.18.2

# Install Go and dev dependencies
RUN set -euo pipefail \
  && cd /tmp \
  && LATEST_GO_VERSION=$(curl --fail --silent --show-error --location "https://golang.org/VERSION?m=text") \
  && if [[ "${GO_VERSION}" = "latest" ]] ; then \
        GO_VERSION_TO_USE="${LATEST_GO_VERSION}" ; \
     else \
        GO_VERSION_TO_USE="go${GO_VERSION}" ; \
     fi \
  && curl --fail --silent --show-error --location "https://go.dev/dl/${GO_VERSION_TO_USE}.${RECODE_INSTANCE_OS}-${RECODE_INSTANCE_ARCH}.tar.gz" --output go.tar.gz \
  && sudo tar --directory /usr/local --extract --file go.tar.gz \
  && rm go.tar.gz \
  && /usr/local/go/bin/go install golang.org/x/tools/cmd/goimports@latest \
  && /usr/local/go/bin/go install github.com/google/wire/cmd/wire@latest \
  && /usr/local/go/bin/go install github.com/golang/mock/mockgen@latest

# Add Go to path
ENV PATH=$PATH:/usr/local/go/bin:$HOME/go/bin

...

💡 What if I don't have an user configuration?

If you don't have an user configuration, the recode-sh/.recode repository will be used as a default one.

That's why you will have zsh configured as default shell in your project.

Recode configuration

As you may have noticed from previous sections, some commands in the dev_env.Dockerfile files (like the LABEL ones) are specific to Recode. This section will try to explain them.

Base image (recode-sh/base-dev-env)

As you may have understood, all the development environments derive directly or indirectly from recode-sh/base-dev-env. You could see the source of this Docker image in the recode-sh/base-dev-env repository:

# All development environments will be Ubuntu-based
FROM ubuntu:22.04

ARG DEBIAN_FRONTEND=noninteractive

# RUN will use bash
SHELL ["/bin/bash", "-c"]

# We want a "standard Ubuntu"
# (ie: not one that has been minimized
# by removing packages and content
# not required in a production system)
RUN yes | unminimize

# Install system dependencies
RUN set -euo pipefail \
  && apt-get --assume-yes --quiet --quiet update \
  && apt-get --assume-yes --quiet --quiet install \
  apt-transport-https \
  build-essential \
  ca-certificates \
  curl \
  git \
  gnupg \
  locales \
  lsb-release \
  man-db \
  manpages-posix \
  nano \
  sudo \
  tzdata \
  unzip \
  vim \
  wget \
  && rm --recursive --force /var/lib/apt/lists/*

# Install the Docker CLI. 
# The Docker daemon socket will be mounted from instance.
RUN set -euo pipefail \
  && curl --fail --silent --show-error --location https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor --output /usr/share/keyrings/docker-archive-keyring.gpg \
  && echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release --codename --short) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null \
  && apt-get --assume-yes --quiet --quiet update \
  && apt-get --assume-yes --quiet --quiet install docker-ce-cli \
  && rm --recursive --force /var/lib/apt/lists/*

# Install Docker compose
RUN set -euo pipefail \
  && LATEST_COMPOSE_VERSION=$(curl --fail --silent --show-error --location "https://api.github.com/repos/docker/compose/releases/latest" | grep --only-matching --perl-regexp '(?<="tag_name": ").+(?=")') \
  && curl --fail --silent --show-error --location "https://github.com/docker/compose/releases/download/${LATEST_COMPOSE_VERSION}/docker-compose-$(uname --kernel-name)-$(uname --machine)" --output /usr/libexec/docker/cli-plugins/docker-compose \
  && chmod +x /usr/libexec/docker/cli-plugins/docker-compose

# Install entrypoint script
COPY ./recode_entrypoint.sh /
RUN chmod +x /recode_entrypoint.sh

# Configure the user "recode" in container.
# Triggered during build on instance.
# 
# We want the user "recode" inside the container to get 
# the same permissions than the user "recode" in the instance 
# (to access the Docker daemon, SSH keys and so on).
# 
# To do this, the two users need to share the same UID/GID.
ONBUILD ARG RECODE_USER_ID
ONBUILD ARG RECODE_USER_GROUP_ID
ONBUILD ARG RECODE_DOCKER_GROUP_ID

ONBUILD RUN set -euo pipefail \
  && RECODE_USER_HOME_DIR="/home/recode" \
  && RECODE_USER_WORKSPACE_DIR="${RECODE_USER_HOME_DIR}/workspace" \
  && RECODE_USER_WORKSPACE_CONFIG_DIR="${RECODE_USER_HOME_DIR}/.workspace-config" \
  && groupadd --gid "${RECODE_USER_GROUP_ID}" --non-unique recode \
  && useradd --gid "${RECODE_USER_GROUP_ID}" --uid "${RECODE_USER_ID}" --non-unique --home "${RECODE_USER_HOME_DIR}" --create-home --shell /bin/bash recode \
  && cp /etc/sudoers /etc/sudoers.orig \
  && echo "recode ALL=(ALL) NOPASSWD:ALL" | tee /etc/sudoers.d/recode > /dev/null \
  && groupadd --gid "${RECODE_DOCKER_GROUP_ID}" --non-unique docker \
  && usermod --append --groups docker recode \
  && mkdir --parents "${RECODE_USER_WORKSPACE_CONFIG_DIR}" \
  && mkdir --parents "${RECODE_USER_WORKSPACE_DIR}" \
  && mkdir --parents "${RECODE_USER_HOME_DIR}/.ssh" \
  && mkdir --parents "${RECODE_USER_HOME_DIR}/.gnupg" \
  && mkdir --parents "${RECODE_USER_HOME_DIR}/.vscode-server" \
  && chown --recursive recode:recode "${RECODE_USER_HOME_DIR}" \
  && chmod 700 "${RECODE_USER_HOME_DIR}/.gnupg"

ONBUILD WORKDIR /home/recode/workspace
ONBUILD USER recode

ONBUILD ENV USER=recode
ONBUILD ENV HOME=/home/recode
ONBUILD ENV EDITOR=/usr/bin/nano

ONBUILD ENV RECODE_WORKSPACE=/home/recode/workspace
ONBUILD ENV RECODE_WORKSPACE_CONFIG=/home/recode/.workspace-config

# Only for documentation purpose.
# Entrypoint and CMD are always set by the 
# Recode agent when running the dev env container.
ONBUILD ENTRYPOINT ["/recode_entrypoint.sh"]
ONBUILD CMD ["sleep", "infinity"]

# Set default timezone
ENV TZ=America/Los_Angeles

# Set default locale
# /!\ locale-gen must be run as root
RUN locale-gen en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en
ENV LC_ALL=en_US.UTF-8

As you can see, nothing fancy here.

Recode is built on ubuntu with docker and docker compose pre-installed. An user recode is created and configured to be used as the default user. Root privileges are managed via sudo.

Your repositories will be cloned in /home/recode/workspace. A default timezone and locale are set.

(To learn more, see the recode-sh/base-dev-env repository).

Visual Studio Code extensions

In order to require Visual Studio Code extensions to be installed in your development environment, you need to add a LABEL named sh.recode.vscode.extensions in your user's or project's dev_env.Dockerfile.

(As you may have guessed, if this label is added to your user configuration, all your projects will have the listed extensions installed).

An extension is identified using its publisher name and extension identifier (publisher.extension). You can see the name on the extension's detail page.

Example

LABEL sh.recode.vscode.extensions="golang.go, zxh404.vscode-proto3, ms-azuretools.vscode-docker"

Multiple repositories

If you want to use multiple repositories in your development environment, you need to add a LABEL named sh.recode.repositories in your project's dev_env.Dockerfile.

In this case, we recommend you to create an empty repository that will only contain the .recode directory (as an example, see the recode-sh/workspace repository).

(As you may have guessed, if this label is added to your user configuration it will be ignored).

Repositories may be set as relative to the current one (eg: cli) or fully qualified (eg: recode-sh/cli).

Example

LABEL sh.recode.repositories="cli, agent, recode, aws-cloud-provider, base-dev-env, api, .recode, workspace"

Build arguments (RECODE_INSTANCE_OS and RECODE_INSTANCE_ARCH)

Given the nature of this project, you need to take into account the fact that the characteristics of the instance used to run your development environment may vary depending on the one chosen by the final user.

As an example, an user may want to use an AWS graviton powered instance to run your project and, as a result, your project's dev_env.Dockerfile must be ready to be built for ARM.

To ease this process, Recode will pass to your dev_env.Dockerfile files two build arguments RECODE_INSTANCE_OS and RECODE_INSTANCE_ARCH that will contain both the current operating system (linux) and architecture (eg: amd64) respectively.

Example

# Reserved args (RECODE_*). Provided by Recode.

# eg: linux
ARG RECODE_INSTANCE_OS

# eg: amd64 or arm64
ARG RECODE_INSTANCE_ARCH

Hooks

Hooks are shell scripts that will be run during the lifetime of your development environment. To be able to add a hook in a project, all you have to do is to add a directory named hooks in your .recode directory.

First Hook

Before adding your first hook, the following things must be taken into account:

• In the case of development environments with only one repository, hooks will only be run if a dev_env.Dockerfile file is set.

• In the case of development environments with multiple repositories, all the hooks will be run, one after the other.

• The working directory of your scripts will be set to the root folder of their respective repository before running.

Init

The init hook is run once, during the first start of your development environment. You could use it to download your project dependencies, for example.

Currently, it's the sole hook available. To activate it, you need to add an init.sh file in your project's hooks directory.

Example (taken from the recode-sh/cli repository)

#!/bin/bash
set -euo pipefail

log () {
  echo -e "${1}" >&2
}

log "Downloading dependencies listed in go.mod"

go mod download

Frequently asked questions

> How does it compare with GitPod / Coder / Codespaces / X?

• 100% Free.
• 100% Open-source.
• 100% Private (run on your own cloud provider account).
• 100% Cost-effective (run on simple VMs not on Kubernetes).
• 100% Desktop.
• 100% Multi regions.
• 100% Customizable (from VM characteristics to installed runtimes).
• 100% Community-driven (see below).

... and 0% VC-backed. 0% Locked-in. 0% Proprietary config files.

> How does it compare with Vagrant / VSCode remote SSH / Container extensions?

• Remote development environments defined as code (with support for user and project configuration).
• Automatic infrastructure / VM provisionning for multiple cloud providers.
• Fully integrated with GitHub (private and multiple repositories, verified commits...).
• Support the pre-installation of VSCode extensions.
• Doesn't require a VM or Docker to be installed locally.
• Doesn't tied to a specific code editor.

> Why using Docker as a VM and not something like NixOS, for example?

I'm aware that containers are not meant to be used as a VM like that (forgive me for that 🙏) but, at the time of writing, Docker is still the most widely used tool among developers to configure their environment (even if it may certainly change in the future).

> Given that my dev env will run in a container does it mean that it will be limited?

Mostly not.

Given the scope of this project (a private instance running in your own cloud provider account), Docker is mostly used for configuration purpose and not to "isolate" the VM from your environment.

As a result, your development environment container will run in privileged mode in the host network.

The future

This project is 100% community-driven, meaning that except for bug fixes no more features will be added.

The only features that will be added are the ones that will be posted as an issue and that will receive a significant amount of upvotes (>= 10 currently).

License

Recode is available as open source under the terms of the MIT License.