Filebeat file rotation problem

[smaniero]

I test file rotation feature of filebeat, and I can't understand how it work.

My config file is:

filebeat:
  prospectors:
    -
      paths:
        - "/var/log/test/*.log"
  registry_file: "registry"

output:
  console:
    pretty: true

Create log file with commands:

echo START > logfile.log
echo ROW 1 >> logfile.log
echo ROW 2 >> logfile.log

Registry file content:
{"/var/log/test/logfile.log":{"source":"/var/log/test/logfile.log","offset":18,"FileStateOS":{"inode":1230350,"device":2065}}}

Rotate log file with command:
mv /var/log/test/logfile.log /var/log/test/logfile.1.log

Repeat command:

echo START > logfile.log
echo ROW 3 >> logfile.log
echo ROW 4 >> logfile.log
echo ROW 5 >> logfile.log

Registry file content:
{"/var/log/test/logfile.log":{"source":"/var/log/test/logfile.log","offset":24,"FileStateOS":{"inode":1230350,"device":2065}}}

After rotation of log file with command "mv", in registry file there's no entry for file "logfile.1.log", and when append lines to logfile.log or logfile1.log in registry single entry offset info change in both cases.

When restart filebeat command, in stdout replay all entries of logfile.1.log and I have duplicated log entry in my repository.

How to solve the problem?
My configuration or use of filebeat is wrong?

[ruflin]

Which version of filebeat are you using?

[smaniero]

Beats 1.1 build on Linux 64bit

[ruflin]

@smaniero Thanks a lot for the detailed report and sort for the late answer. Unfortunately I was able to reproduce it and it is indeed a bug. Here is the fix for it: #1060

[ruflin]

Backport fix for 1.1 can be found here: #1061