UUID assigned to enrolled Windows Filebeat agent not matching one in Central Management Console

[jasonboshears]

Hello, I upgraded our ELK instance to 6.5.1 and installed the Windows Filebeat 6.5.1 agent on a Windows system I wanted to monitor. All seemed to be running fine. I decided to try out the new Filebeat centralized management feature and enrolled the Windows Filebeat client on the server. The Windows client was able to enroll fine, but no further contact was made, and configurations were not updated as expected. Upon further investigation, I saw this logged on the client: error retriving new configurations, will use cached ones: Beat "xxxxxxxx" not found. I then noticed the UUID stored in the meta.json file is different from the one shown in Central Management Console. When I updated the meta.json file with the UUID listed in Central Management and restarted services, all seemed to work as expected. I was able to reproduce this behavior consistently on six systems I tried to enroll in Central Management.

I am able to work around this error by manually updating the UUID in meta.json on the client system. It is an inconvenience but not critically impactful. However, I think I may have found a couple bugs in the new version of the Windows Filebeat client. There appears to be an inconsistent UUID being assigned to server and client, as well as the misspelling of 'retrieving' in the error log.

[ph]

@jasonboshears I haven't seen this behavior before. You are saying that you can reproduce this issue everytime? Generating the UUID is one of the first thing we do when we start the libbeat framework and it done synchronously.

Can you try the following:

• At the beginning do the data/meta.json file exist?
• After running the enroll command what is the content of the data/meta.json
• The uuid doesn't match the one in the Central management UI?

[nickerber]

@ph I'm having the same problem, already opened a discussion here: https://discuss.elastic.co/t/central-management-wrong-beat-id/160498, where I described my method.
Using the filebeat and metricbeat in it's version 6.5.3. After enrolling, the meta.json is generated with a different ID than the one in Central Management.

[nickerber]

I think I found the problem.
After firing the enroll command on one of the clients there is a data/meta.json created which is registered against the Management UI (because these two IDs fit!)
But when starting the beat as a service it seems to ignore this data/meta.json and, instead, it generates another meta.json in %PROGRAMDATA%/filebeat with a completely different ID, because by default path.data of the service points to C:\ProgramData\filebeat.
So the beat takes the ID in Programdata and of course, doesn't find the counter part in Kibana.

• PROGRAMDATA
  • filebeat
    -meta.json <-- wrong ID
• PROGRAMFILES
  • filebeat
    • data
      • meta.json <-- correct ID, but won't be used from service by default

[ph]

@nickerber That's a good catch! Thanks this give us what we need to fix it.

[ph]

@nickerber sorry for not getting back to you sooner, I don't think we can solve it directly by code, instead we should provide better documentations.

When you install Filebeat as a service using the install-service-metricbeat.ps1 we take care of configuring a few things when we starts the beat. The most important thing is that we set the PATH of the data folder.

  -binaryPathName "`"$workdir\metricbeat.exe`" -c `"$workdir\metricbeat.yml`" -path.home `"$workdir`" -path.data `"C:\ProgramData\metricbeat`" -path.logs `"C:\ProgramData\metricbeat\logs`""

So this explains why the enroll command that you execute directly and the one that the service starts doesn't use the same UUID. By default the beat lookup in the current folder.

If you add the -path.data to the enroll subcommand it should use the same UUID.

.\filebeat.exe enroll http://xyz.gov:5601 70f4b584e8024b96b682c46125a8d81a --username elastic --password stdin --force -path.data "C:\ProgramData\filebeat"

[ph]

I will close this issue and open a one to improve documentation with Windows.

[ph]

Followup issue at #10415