[BUG] Docker & Docker compose installed from Homebrew not recognized

[lukas-vlcek]

Describe the bug

I am trying to setup dev environment on Mac M1 and I have docker installed from Homebrew (brew install docker docker-compose;). When I start assemble task I get an exception about missing Docker binaries like this:

Docker (checked [/usr/bin/docker, /usr/local/bin/docker]) is required to run the following tasks: 

I think the issue is that brew by default installs binaries into different location. See:

% which docker docker-compose
/opt/homebrew/bin/docker
/opt/homebrew/bin/docker-compose

# Just FYI my version of Docker-compose and Docker

% docker-compose version
Docker Compose version 2.5.1

% docker version
Client: Docker Engine - Community
 Version:           20.10.16
 API version:       1.41
 Go version:        go1.18.2
 Git commit:        aa7e414fdc
 Built:             Wed May 11 16:22:17 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.14
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.15
  Git commit:       87a90dc
  Built:            Thu Mar 24 01:48:38 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          v1.5.11
  GitCommit:        3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc:
  Version:          1.0.3
  GitCommit:        f46b6ba2c9314cfc8caae24a32ec5fe9ef1059fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The exception seems to be coming from DockerSupportService.java

OpenSearch/buildSrc/src/main/java/org/opensearch/gradle/docker/DockerSupportService.java

Line 165 in e9b19a0

"Docker (checked [%s]) is required to run the following task%s: \n%s",

It does not currently check for Docker binaries in the location where brew installs it.

To Reproduce
Steps to reproduce the behavior:

A fresh pull from main:

% git pull upstream main
From github.com:opensearch-project/OpenSearch
 * branch                    main       -> FETCH_HEAD
Already up to date.

% git log --oneline -n 1
e9b19a095c2 (HEAD -> main, upstream/main, lukas/main) [Segment Replication] Add SegmentReplicationTargetService to orchestrate replication events. (#3439)

Run assemble task.

% ./gradlew assemble    

> Configure project :qa:os
Cannot add task 'destructiveDistroTest.docker' as a task with that name already exists.
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 7.4.2
  OS Info               : Mac OS X 12.3.1 (aarch64)
  JDK Version           : 17 (Eclipse Temurin JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home
  Random Testing Seed   : 86D4E779628BC837
  In FIPS 140 mode      : false
=======================================

FAILURE: Build failed with an exception.

* What went wrong:
Docker (checked [/usr/bin/docker, /usr/local/bin/docker]) is required to run the following tasks: 
:distribution:docker:buildArm64DockerImage
:distribution:docker:buildDockerImage
you can address this by attending to the reported issue, or removing the offending tasks from being executed.

Expected behavior

DockerSupportService.java should be able to detect Docker tools in location used by brew.

Host/Environment (please complete the following information):

• OS: Mac OS, chip M1: % uname -a Darwin Lukass-MacBook-Pro.local 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64

[minalsha]

Assigning it to you @lukas-vlcek since you have a PR already for this issue.

[minalsha]

Hi @lukas-vlcek what is pending on addressing this issue? Thanks

[minalsha]

Hi @lukas-vlcek , reaching out to you again. Are you still working on this and if so, could you please share what is pending on this issue? Thanks

[lukas-vlcek]

@minalsha I haven't been working on this recently. I can have a look at it starting on Mar 20, when I am back from PTO.
I think this issue hasn't been a real issue for many people because I haven't seen any follow-up or questions on this thread. Probably it is a very low priority.

[lukas-vlcek]

I suggest closing this ticket, as well as #3481
I prefer this solution: #6877

[dblock]

@lukas-vlcek Maybe we should keep this open since the issue is still there? we can close when we actually fix the problem one way or another?

[lukas-vlcek]

@dblock Sure, we can keep this open for now.

But just for the context, this path is not what would allow me to run full OpenSearch testing suite including tests that require docker/docker-compose, because I would need to install something that is not fully OSS or require license. My only alternative is going the "podman" direction.

[dblock]

@lukas-vlcek I see, so your biggest objection is docker non-OSS? I would make it clear in the podman proposal, I would support it.