bump python versions and use docker slim versions

[v1v]

What does this pull request do?

Use smaller docker images (slim) and bump the version to use the supported python version for some of the scripts

Why

vs

Then the docker image with all the tooling is about 2x smaller:

REPOSITORY TAG IMAGE ID CREATED SIZE
apm-agent-python python-3.6 50ecd8523761 51 seconds ago 462MB

vs

REPOSITORY TAG IMAGE ID CREATED SIZE
apm-agent-python python-3.6 8e23ab62a462 16 seconds ago 961MB

Implementation details

There were some missing packages required to run some of the tests, such as:

• libmariadb-dev to fix mariadb_config: not found when using the framework mysqlclient-newest
• libpq-dev to fix pg_config executable not found when using the framework psycopg2-newest
• build-essential to fix gcc executable not found when using the framework twisted
• make to fix the run of all the tests.
• gpg to fix the docker image generation.

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-10-06T14:56:04.102+0000

• Duration: 38 min 42 sec

Test stats 🧪

Test	Results
Failed	0
Passed	13254
Skipped	15092
Total	28346

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /test linters : Run the Python linters only.

• /test full : Run the full matrix of tests.

• /test benchmark : Run the APM Agent Python benchmarks tests.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[apmmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (67/67)	💚
Files	100.0% (226/226)	💚
Classes	100.0% (226/226)	💚
Lines	90.657% (17902/19747)	👎 -0.643
Conditionals	74.967% (2803/3739)	👍 0.36

[v1v]

I'll run a few builds in the CI to discard any issues with the full test coverage

[v1v]

/test full

[v1v]

/test full

[amannocci]

LGTM

Small suggestion :

# Use --no-cache-dir flag.
# pre-install base requirements
RUN python -m pip install --no-cache-dir --progress-bar off -U pip && pip install --no-cache-dir --progress-bar off -r /reqs-base.txt

[beniwohli]

I wonder if these scripts are still in use. We use pre-commit for black/isort/flake8. The only reference I could find is in run-tests-locally.asciidoc

From what I can tell, we might as well just delete these three scripts.

[v1v]

Good point, let me review it in a follow up, as I don't wanna change any further behaviours in this PR but only bump the version to use a slim one and default to a major python version in those scripts.

[v1v]

Small suggestion :

Thanks @amannocci , this change was not aimed to modify any of the internals but the docker image base image and default python version, for now I'll not make any further changes so this can be merged and in a follow up the implementation can be revisited 🙏