Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Agent] Docker image for integration tests #16898

Merged
merged 6 commits into from
Mar 16, 2020

Conversation

michalpristas
Copy link
Contributor

Newly created container definition for agent enrolls to kibana first and then runs and waits for kibana to provide configuration.

Configuration uses variables to connect to services with fallbacks to localhost on standard ports. This enables you to spin up kibana/es and start agent in a container running

docker build -t agent .  
docker run --network host agent

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Project:fleet)

@@ -1,6 +1,9 @@
# agent
build/
agent.dev.yml
cmd/agent/agent
cmd/agent/agent.yml
cmd/agent/fleet.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why theses files are created?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are created when you build and run agent there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm usually building latest in x-pack/agent/cmd/agent and enrolling to fleet. this produces some garbage i don't want to check in

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually use mage build:all at the root :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe its would be better to have them defined in your local ignore?

setup
enroll

exec {{ .BeatName }} run "$@"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a small problem here, for testing using a docker container this work great, the setup + enroll using credentials but in the normal scenario we would use an enrollment key and nothing else, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this container serves testing first, to be usable in a normal use cases it needs some tweaking. i was thinking about different options such as using flags to determine if we're testing or not or setting env variable like SETUP_KIBANA or PERFORM_ENROLL to perform any additional steps. without variables it would just do run.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have mixed feeling here, having something quickly working and adding a bit more work to make it behave with enroll? Do you think it is a lot of work to add it now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no i dont' but i'm not sure about which approach to take. flags vs env vars vs something else i dont see

Copy link
Contributor

@ph ph Mar 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think environment variables is fine for first version, I prefer to have the functionality right and we can change the flags until we move to GA.

@ph ph added the needs_backport PR is waiting to be backported to other branches. label Mar 10, 2020
@michalpristas
Copy link
Contributor Author

michalpristas commented Mar 11, 2020

Updated with env variables, to make docker run (locally) everything for you with default usernames and paths you need to do: docker run --network host -e FLEET_SETUP=1 -e FLEET_ENROLL=1 agent after docker build

if you start container without any env vars, simple agent run is performed

@ph
Copy link
Contributor

ph commented Mar 11, 2020

Works for me.

Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@michalpristas
Copy link
Contributor Author

Jenkins test this

@michalpristas michalpristas merged commit 49fe4c9 into elastic:master Mar 16, 2020
@@ -160,7 +160,7 @@ shared:
<<: *agent_binary_spec
extra_vars:
from: 'centos:7'
user: '{{ .BeatName }}'
user: 'root'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalpristas @ph @andrewkroh @urso Just stumbled over this change. I wonder if this could become a problem or is ok?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one was done on purpose as agent needs root permissions, the specification here is used for agent only so no beat is affected

ph pushed a commit to ph/beats that referenced this pull request Apr 7, 2020
@ph ph removed the needs_backport PR is waiting to be backported to other branches. label Apr 7, 2020
@ph
Copy link
Contributor

ph commented Apr 7, 2020

backported in #17574 to have a working build

ph added a commit that referenced this pull request Apr 7, 2020
… enable building the agent. (#17574)

* Fix issue when building Agent snapshot and enable building the agent. (#16927)

An extension in the generated files ("SNAPSHOT") was preventing us to
correctly generate the agent artifact with the packaged
metricbeat/filebeat.

Fixes: #16908
(cherry picked from commit 62d3683)

* [Agent] Fix installers (#17077)

[Agent] Fix installers for snapshots (#17077)

* [Agent] fixed mage package for agent (#17368)

[Agent] fixed mage package for agent (#17368)

* [Agent] Make default config OS agnostic (#17016)

[Agent] Make default config OS agnostic (#17016)

* [Agent] Docker image for integration tests (#16898)

[Agent] Docker image for integration tests (#16898)

Co-authored-by: Michal Pristas <michal.pristas@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants