Skip to content

Commit

Permalink
Cherry-pick #19960 to 7.x: [ci] Favor direct mage invocation on CI (#…
Browse files Browse the repository at this point in the history
…20030)

* [ci] Favor direct mage invocation on CI (#19960)

This changes Jenkins and Travis to directly invoke mage where
possible instead of going through make. Some of the remaining
make don't yet have a mage equivalant (mainly crosscompile).

For Packetbeat this add Jenkins stages to test on darwin and Windows.

There were a few fixes I had to make related to these changes:

- Add some mage target dependencies to ensure fields and dashboards
  are ready when tests use them.
- Swap the order of the Go imports for dev-tools/mage/targets/integtest
  and unittest so that unit tests run before integ tests when running
  the 'mage test' target.
- chown the shared Python venv that is in the root of the repo after
  Dockerized integ tests exit to ensure there are no permissions issues
  caused by root owned files.
- I found a few Python string encoding issues that caused tests failures.
  I thought we fixed these during the Python 3 conversion, but something
  here exposed a few that we didn't address.

One thing of note that I did not correct. Journalbeat has system tests
but they are not executed. The existing/old Makefile has SYSTEM_TESTS=false
so they are not executed. So when I switched it to mage I left it as
'mage goUnitTest' to avoid the Python when failed when I tried it.

(cherry picked from commit 4a1f800)
  • Loading branch information
andrewkroh authored Jul 29, 2020
1 parent 7e884d9 commit 4c6dba2
Show file tree
Hide file tree
Showing 15 changed files with 249 additions and 73 deletions.
25 changes: 25 additions & 0 deletions .ci/scripts/travis_has_changes.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -exuo pipefail

# Changes on these files will trigger all builds.
COMMON_DIRLIST="dev-tools .travis.yml testing .ci"

# Commit range to check for. For example master...<PR branch>
RANGE=$TRAVIS_COMMIT_RANGE
DIRLIST="$@ $COMMON_DIRLIST"

# Find modified files in range and filter out docs only changes.
CHANGED_FILES=$(git diff --name-only $RANGE | grep -v '.asciidoc')

beginswith() { case $2 in "$1"*) true;; *) false;; esac }

for path in $DIRLIST; do
for changed in $CHANGED_FILES; do
if beginswith $path $changed; then
exit 0 # found a match -> continue testing
fi
done
done

echo "NOT testing required. Modified files: $CHANGED_FILES"
exit 1
169 changes: 136 additions & 33 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,193 +37,287 @@ jobs:

# Filebeat
- os: linux
env: TARGETS="-C filebeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh filebeat libbeat || travis_terminate 0
env:
- PROJECT=filebeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
env: TARGETS="TEST_ENVIRONMENT=0 -C filebeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh filebeat libbeat || travis_terminate 0
env:
- PROJECT=filebeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/filebeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh x-pack/filebeat filebeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/filebeat
- MAGE='build test'
go: $(GO_VERSION)
stage: test

# Heartbeat
- os: linux
env: TARGETS="-C heartbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh heartbeat libbeat || travis_terminate 0
env:
- PROJECT=heartbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
env: TARGETS="TEST_ENVIRONMENT=0 -C heartbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh heartbeat libbeat || travis_terminate 0
env:
- PROJECT=heartbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test

# Auditbeat
- os: linux
env: TARGETS="-C auditbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh auditbeat libbeat || travis_terminate 0
env:
- PROJECT=auditbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
env: TARGETS="TEST_ENVIRONMENT=0 -C auditbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh auditbeat libbeat || travis_terminate 0
env:
- PROJECT=auditbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh auditbeat libbeat || travis_terminate 0
env: TARGETS="-C auditbeat crosscompile"
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/auditbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh x-pack/auditbeat auditbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/auditbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
before_install: .ci/scripts/travis_has_changes.sh x-pack/auditbeat auditbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/auditbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test

# Libbeat
- os: linux
env: TARGETS="-C libbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh libbeat || travis_terminate 0
env:
- PROJECT=libbeat
- MAGE='build test'
# The libbeat tests are so verbose that they exceed the maximum allowed log length of Travis CI.
- MAGEFILE_VERBOSE=false
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh libbeat || travis_terminate 0
env: TARGETS="-C libbeat crosscompile"
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh libbeat || travis_terminate 0
env: STRESS_TEST_OPTIONS="-timeout=20m -race -v -parallel 1" TARGETS="-C libbeat stress-tests"
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/libbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh x-pack/libbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/libbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test

# Metricbeat
- os: linux
env: TARGETS="-C metricbeat unit-tests"
before_install: .ci/scripts/travis_has_changes.sh metricbeat libbeat || travis_terminate 0
env:
- PROJECT=metricbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh metricbeat libbeat || travis_terminate 0
install:
install:
- .ci/scripts/install-kind.sh
- .ci/scripts/install-kubectl.sh
env:
- TARGETS="-C metricbeat integration-tests"
- K8S_VERSION=v1.17.2
- KIND_VERSION=v0.7.0
- PROJECT=metricbeat
- MAGE='goIntegTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C metricbeat system-tests"
before_install: .ci/scripts/travis_has_changes.sh metricbeat libbeat || travis_terminate 0
env:
- PROJECT=metricbeat
- MAGE='pythonIntegTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
env: TARGETS="-C metricbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh metricbeat libbeat || travis_terminate 0
env:
- PROJECT=metricbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh metricbeat libbeat || travis_terminate 0
env: TARGETS="-C metricbeat crosscompile"
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/metricbeat unit-tests"
before_install: .ci/scripts/travis_has_changes.sh x-pack/metricbeat metricbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/metricbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/metricbeat integration-tests"
before_install: .ci/scripts/travis_has_changes.sh x-pack/metricbeat metricbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/metricbeat
- MAGE='goIntegTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/metricbeat system-tests"
before_install: .ci/scripts/travis_has_changes.sh x-pack/metricbeat metricbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/metricbeat
- MAGE='pythonIntegTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
env: TARGETS="-C x-pack/metricbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh metricbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/metricbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test

# Packetbeat
- os: linux
env: TARGETS="-C packetbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh packetbeat libbeat || travis_terminate 0
env:
- PROJECT=packetbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test

# Winlogbeat
- os: linux
before_install: .ci/scripts/travis_has_changes.sh winlogbeat libbeat || travis_terminate 0
env: TARGETS="-C winlogbeat crosscompile"
go: $TRAVIS_GO_VERSION
stage: test

# Functionbeat
- os: linux
env: TARGETS="-C x-pack/functionbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh x-pack/functionbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/functionbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
env: TARGETS="TEST_ENVIRONMENT=0 -C x-pack/functionbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh x-pack/functionbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/functionbeat
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh x-pack/functionbeat libbeat || travis_terminate 0
env: TARGETS="-C x-pack/functionbeat test-gcp-functions"
env:
- PROJECT=x-pack/functionbeat
- MAGE='testGCPFunctions'
go: 1.13.1
stage: test

# Docker Log Driver
- os: linux
env: TARGETS="-C x-pack/dockerlogbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh x-pack/dockerlogbeat libbeat || travis_terminate 0
env:
- PROJECT=x-pack/dockerlogbeat
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test

# Journalbeat
- os: linux
env: TARGETS="-C journalbeat testsuite"
before_install: .ci/scripts/travis_has_changes.sh journalbeat libbeat || travis_terminate 0
env:
- PROJECT=journalbeat
- MAGE='build goUnitTest'
go: $TRAVIS_GO_VERSION
stage: test

# Agent
- os: linux
before_install: .ci/scripts/travis_has_changes.sh x-pack/elastic-agent libbeat || travis_terminate 0
env: TARGETS="-C x-pack/elastic-agent testsuite"
env:
- PROJECT=x-pack/elastic-agent
- MAGE='build test'
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
before_install: .ci/scripts/travis_has_changes.sh x-pack/elastic-agent libbeat || travis_terminate 0
env: TARGETS="TEST_ENVIRONMENT=0 -C x-pack/elastic-agent testsuite"
env:
- PROJECT=x-pack/elastic-agent
- MAGE='build unitTest'
go: $TRAVIS_GO_VERSION
stage: test

# Generators
- os: linux
before_install: .ci/scripts/travis_has_changes.sh generator metricbeat libbeat || travis_terminate 0
env: TARGETS="-C generator/_templates/metricbeat test test-package"
go: $TRAVIS_GO_VERSION
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh generator libbeat || travis_terminate 0
env: TARGETS="-C generator/_templates/beat test test-package"
go: $TRAVIS_GO_VERSION
stage: test

- os: osx
before_install: .ci/scripts/travis_has_changes.sh generator metricbeat libbeat || travis_terminate 0
env: TARGETS="-C generator/_templates/metricbeat test"
go: $TRAVIS_GO_VERSION
stage: test
- os: osx
before_install: .ci/scripts/travis_has_changes.sh generator libbeat || travis_terminate 0
env: TARGETS="-C generator/_templates/beat test"
go: $TRAVIS_GO_VERSION
stage: test

# Kubernetes
- os: linux
before_install: .ci/scripts/travis_has_changes.sh deploy/kubernetes metricbeat filebeat libbeat || travis_terminate 0
install: deploy/kubernetes/.travis/setup.sh
env:
- TARGETS="-C deploy/kubernetes test"
- TRAVIS_K8S_VERSION=v1.9.4
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh deploy/kubernetes metricbeat filebeat libbeat || travis_terminate 0
install: deploy/kubernetes/.travis/setup.sh
env:
- TARGETS="-C deploy/kubernetes test"
- TRAVIS_K8S_VERSION=v1.10.0
stage: test
- os: linux
before_install: .ci/scripts/travis_has_changes.sh deploy/kubernetes metricbeat filebeat libbeat || travis_terminate 0
dist: xenial
install: deploy/kubernetes/.travis/setup.sh
env:
Expand Down Expand Up @@ -267,7 +361,12 @@ addons:
- python3.6
- python3.6-venv

before_install:


# Skips installations step
install: true

before_script:
- if [ x$TRAVIS_DIST = xtrusty ]; then sudo ln -sf python3.6 /usr/bin/python3; fi
- python --version
- python3 --version
Expand All @@ -279,17 +378,21 @@ before_install:
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
- if [ $TRAVIS_OS_NAME = osx ]; then pip install virtualenv==16.7.9; fi


# Skips installations step
install: true
- make mage

script:
# Replacement for travis_wait which doesn't print output in real time.
# Default Travis timeout is 10min, so this workaround prints timestamps every 9min to reset the counter.
# Using seconds (540s = 9min) instead of minutes for shell compatibility reasons.
- while sleep 540; do echo "=====[ ${SECONDS} seconds still running ]====="; done &
- make $TARGETS
- if [[ -n "$MAGE" ]]; then
echo ">> mage $MAGE from $PROJECT";
cd "$PROJECT";
mage $MAGE;
else
echo ">> make $TARGETS";
make $TARGETS;
fi
- kill %1

notifications:
Expand Down
Loading

0 comments on commit 4c6dba2

Please sign in to comment.