Add close_removed and close_renamed, deprecate force_close_files #1909
Conversation
force_close_files is replaced by the two option close_removed and close_renamed. Force_close_files is deprecated. In case it is enabled, it sets close_removed and close_renamed to true. This is part of elastic#1600
| # file is skipped, as the reading starts at the end. We recommend to leave this option on false | ||
| # but lower the ignore_older value to release files faster. | ||
| #force_close_files: false | ||
| # Close renamed means a file handler is close when a file is renamed / rotated. In case the harvester was |
There was a problem hiding this comment.
'Close ensures a file handler is closed when the file is renamed or rotated'.
'Note: Potential data loss if renamed file is not picked up by prospector'.
There was a problem hiding this comment.
Added accordingly, and move the change to beat.yml :-)
| if r.config.ForceClose { | ||
| // Check if the file name exists (see #93) | ||
| if r.config.CloseRenamed { | ||
| if !file.IsSameFile(r.fs.Name(), info) { |
There was a problem hiding this comment.
let's keep the code comment Check if the file name still exists.
There was a problem hiding this comment.
Shouldn't it be: check if file has been renamed, but still exists ?
There was a problem hiding this comment.
I added Check if the file has been renamed. It does not check if the file still exists, it only checks if the file with the same name is this file or an other file. In case no file under the same name exists, it just assumes rename (it could also be removed). But that is checked in the next check.
There was a problem hiding this comment.
Sorry, changed it again to: // Check if the file can still be found under the same path
|
@urso review applied and new version pushed. |
force_close_files is replaced by the two option close_removed and close_renamed. Force_close_files is deprecated. In case it is enabled, it sets close_removed and close_renamed to true.
This is part of #1600