Adding php-fpm module to metricbeat

[hartfordfive]

I'm currently working on adding both of these modules to metricbeats although I figured i'd create an issue for this so the community can see it's currently in progress in case anyone is interested in using them.

[ruflin]

@hartfordfive Great to hear you are working on these modules. Please let us know if hit some roadblocks or need some inputs.

[hartfordfive]

@ruflin So i'm trying to create and run the necessary tests for the haproxy module although i'm getting this error when running the make integration-tests-environment command:

docker-compose -f /Users/al/Documents/code/github/elastic-beats/metricbeat/../testing/environments/base.yml -f /Users/al/Documents/code/github/elastic-beats/metricbeat/../testing/environments/snapshot.yml -f docker-compose.yml run beat make integration-tests RACE_DETECTOR=
Starting elasticsearch
ERROR: Cannot start service beat: mkdir /Users/al/Documents/code/github/elastic-beats: file exists
make: *** [integration-tests-environment] Error 1

I'm running the command in the local directory "/Users/al/Documents/code/github/elastic-beats/metricbeat" Any ideas exactly what it is that I'm I doing wrong here? You can find my current code here: https://github.com/hartfordfive/beats/tree/metricbeat-haproxy

[ruflin]

@hartfordfive I think the issue is with your GOPATH setup. Golang requires you to have the project in the correct directory of your $GOPATH. So in this case the project must be under $GOPATH/src/github.com/elastic/beats otherwise it will not work.

[hartfordfive]

@ruflin Sorry I forgot to mention something. My current path /Users/al/Documents/code/github/elastic-beats/metricbeat is actually just a sym link to the correct project Go directory /usr/local/go/src/github.com/elastic/beats/. This made things a bit more convenient for me in my IDE.

[ruflin]

@hartfordfive TBH I don't know what golang does with the symlinks and especially our Makefile magic. Could you try to run the commands in the directory itself?

[hartfordfive]

@ruflin I've tried what you've suggested and had some progress although now I'm getting new errors. First, it seems like the test just waits for HAProxy to start, but it never ends up starting.

Waiting for HAProxy(haproxy:14567) to start..............................................
HAProxy is not available
Address: haproxy:14567

I'm not sure why this is happening as I've tested locally with /metricbeat -e -d "*", along with a local running instance of HAProxy with the proper config, and it works fine.

Secondly, I'm getting errors saying that it can't find some specific Go libraries ("github.com/mitchellh/mapstructure" and "github.com/gocarina/gocsv"), although they are installed as I can test locally just fine. Here's a gist link to the full output of the make command:
https://gist.github.com/hartfordfive/dfe054c1957a981fa6fc503d6fb0286c

Once I can get these issues resolved, I think I should be relatively close to having completed the HAProxy module.

[ruflin]

Which Golang version are you using? Are the most recent changes (which are not working) here: https://github.com/hartfordfive/beats/tree/metricbeat-haproxy If yes, I could directly check out the branch and play around with it to find the problem.

[hartfordfive]

I'm using Go 1.6.2 (darwin/amd64) and yes the most recent changes have been pushed to my branch.

[hartfordfive]

I've actually been able to resolve the issue with the missing libraries by installing them to the vendor/ directory with the govendor fetch command.

[ruflin]

@hartfordfive The strange part is that if you cloned the beats repo, all the dependencies are actually checked in and should already be in the vendor directory. I'm glad you got it working now. Is there something more I could help with? Feel free to open a PR early to get feedback on it or also for questions.

[hartfordfive]

@ruflin Regarding the missing libraries, they are new ones that weren't previously in the vendor directory. I've created PR #2384 although I'm still having the issue with the testing script waiting HAProxy process to start on the docker instance. Is there a special config that I might be missing? I've created the necessary test config and pointed to it in the Dockerfiler (module/haproxy/_meta/haproxy.conf).

[ruflin]

I just checked out your branch (did some merging) and run make integration-tests-environment. It all works on my side. Perhaps you need to stop / clean containers again and restart the tests?

[ruflin]

@hartfordfive I changed the title of this issue to only mention php-fpm as haproxy is now already in metricbeat.

[monicasarbu]

Closing as #3415 was merged