initializeCommand should run before docker-compose config

[davidtlascelles]

Type: Bug

Our team uses the initializeCommand: devcontainer.json key to generate an .env file that is used by the Dev Container. Using the docker-compose env_file: key with a non-existent file causes the docker-compose config command in the startup sequence to fail, even though the .env file will exist by the time the container runs.

I found that I could work around this issue by using the devcontainer.json option: "runArgs": ["--env-file",".devcontainer/.env"], but I believe the real solution is to run the initializeCommand before running any docker related commands, as there could be other reasons the docker-compose config command might fail even though the initializeCommand would have fixed the issues before the container launches.

Interestingly, I found that using the devcontainer CLI tool,
devcontainer up --workspace-folder /path/to/workspace
resulted in the container successfully launching, so it seems that startup sequence is either ordered "correctly" by running the initializeCommand before docker commands, or it just doesn't run a docker-compose config validation at all.

Extension version: 0.255.3
VS Code version: Code 1.72.2 (d045a5eda657f4d7b676dedbfa7aab8207f8a075, 2022-10-12T22:15:18.074Z)
OS version: Windows_NT x64 10.0.19044
Modes:
Sandboxed: No
Remote OS version: Linux x64 5.10.102.1-microsoft-standard-WSL2

System Info

Item	Value
CPUs	11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1498)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	undefined
Memory (System)	15.73GB (0.95GB free)
Process Argv	--crash-reporter-id 217cde51-3d54-4c89-af52-fc539da9dfce
Screen Reader	no
VM	0%

Item	Value
Remote	Dev Container: Codename
OS	Linux x64 5.10.102.1-microsoft-standard-WSL2
CPUs	11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1497)
Memory (System)	12.26GB (10.74GB free)
VM	0%

[chrmarti]

Could you append the log? (F1 > Dev Containers: Show Container Log)

[motiejunas]

Having the same issue. Thanks for runArgs workaround.

[artificial-aidan]

[154261 ms] Start: Run: docker version --format {{.Server.APIVersion}}
[156364 ms] 1.41
[159064 ms] Start: Run: /Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Code - Insiders Helper (Plugin).app/Contents/MacOS/Code - Insiders Helper (Plugin) --ms-enable-electron-run-as-node /Users/aidan/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.262.3/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /Users/aidan/Development/Artificial/artificial-prl-adapter --log-level debug --log-format json --config /Users/aidan/Development/Artificial/artificial-prl-adapter/.devcontainer/devcontainer.json --include-merged-configuration --mount-workspace-git-root true --terminal-columns 246 --terminal-rows 28
[160368 ms] (node:72733) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
[160368 ms] (Use `Code - Insiders Helper (Plugin) --trace-deprecation ...` to show where the warning was created)
[160376 ms] @devcontainers/cli 0.23.2. Node.js v16.14.2. darwin 19.6.0 x64.
[160376 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/Users/aidan/Development/Artificial/artificial-prl-adapter
[162694 ms] Start: Run: docker-compose version --short
[164930 ms] Docker Compose version: 2.6.1
[164930 ms] Start: Run: docker-compose -f /Users/aidan/Development/Artificial/artificial-prl-adapter/.devcontainer/docker-compose.yml --profile * config
[166047 ms] 
[166047 ms] open /Users/aidan/Development/Artificial/artificial-prl-adapter/.devcontainer/test.env: no such file or directory

[166047 ms] Exit code 14
[166048 ms] Error: Command failed: docker-compose -f /Users/aidan/Development/Artificial/artificial-prl-adapter/.devcontainer/docker-compose.yml --profile * config
    at xc (/Users/aidan/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.262.3/dist/spec-node/devContainersSpecCLI.js:1878:904)
    at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async TO (/Users/aidan/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.262.3/dist/spec-node/devContainersSpecCLI.js:1804:20191)
    at async sue (/Users/aidan/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.262.3/dist/spec-node/devContainersSpecCLI.js:2013:31891)
[166090 ms] Exit code 1

[artificial-aidan]

Update: The current workaround I have is to leave the env file blank, and the initialize command will fill it out. Which has its own set of issues because adding an empty file to git and have it not be tracked is another issue.

This actually didn't work. Made a mistake in testing. So as far as I know there isn't a workaround. runArgs didn't work for me, it said it was not an allowed setting.

Going to try and use postCreateCommand

I used a different workaround.

Use a docker-compose override file:

	"dockerComposeFile": [
		"docker-compose.yml",
		"docker-compose.override.yml"
	],
        "initializeCommand": "echo HOST_INFO=$(.devcontainer/get-env.sh) > .devcontainer/host.env",

And set the env file in that.

version: '3'
services:
  adapter:
    env_file: host.env

[chrmarti]

The fix from #7377 is available in Dev Containers 0.264.0-pre-release and later. Please give that a try.

[artificial-aidan]

That appears to have worked. Thank you. I will subscribe to releases to see when I can roll this out to non-pre-release devs.

For anyone else stuck on this my final solution worked well.

docker-compose can use a .env file at a project level.

So my initialize command wrote to .devcontainer/.env which docker-compose picked up automatically.

# .env file
HOST_INFO=hostinfo

version: '3'
services:
  adapter:
    environment:
      # Pull from the project level .env file created by the initializeCommand
      # - HOST_INFO=${HOST_INFO:-}

[github-actions]

Hey @chrmarti, this issue might need further attention.

@davidtlascelles, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[davidtlascelles]

This issue is resolved, thanks.

For posterity: if anyone finds themselves here with a similar issue, it may be related to your version of docker-compose. I found that docker-compose --version 1.25.x (The default for Ubuntu 20.04) has a bug where the .env file in the directory with docker-compose.yml wasn't being loaded when it should be, causing a volume binding issue due to a path not being set from the .env file.

This is fixed by uninstalling the apt version of docker-compose and installing either the docker-compose-plugin (official method per Docker's docs) or the pip-installed version, which is 1.29.x