Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new fingerprint file identity #35734

Merged
merged 22 commits into from
Jul 14, 2023
Merged

Add new fingerprint file identity #35734

merged 22 commits into from
Jul 14, 2023

Commits on Jul 11, 2023

  1. Add new fingerprint file identity 

    This is a new alternative to existing options like `native`, `path`
and `inode_marker`.

Unlike the existing options, this file identity does not rely on any
file system metadata and uses only the file size and its content.

Users can specify what amount of bytes is used to fingerprint the
beginning of each file.

This identity is supposed to be more stable and less affected by the
environment/setup of the users.
    @rdner
    rdner committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    0da3a03 View commit details
    Browse the repository at this point in the history

  2. Fix the integration test

    @rdner
    rdner committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    6603081 View commit details
    Browse the repository at this point in the history

  3. Close file after fingerprinting

    @rdner
    rdner committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    a1c3043 View commit details
    Browse the repository at this point in the history

  4. Move fingerprinting to a lower level (fswatcher) 

    So, we can handle removals and postponed ingestion.

Also, a few performance optimizations included.
    @rdner
    rdner committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    d47387a View commit details
    Browse the repository at this point in the history

  5. Update docs

    @rdner
    rdner committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    07c5511 View commit details
    Browse the repository at this point in the history

  6. Some fixes after rebase and switch of approach

    @rdner
    rdner committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    2be9005 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Add fswatch tests and make improvements

    @rdner
    rdner committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    e86accb View commit details
    Browse the repository at this point in the history

  2. Update reference

    @rdner
    rdner committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    76c8943 View commit details
    Browse the repository at this point in the history

  3. Add the changelog entry

    @rdner
    rdner committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    e20d1f9 View commit details
    Browse the repository at this point in the history

  4. Make update

    @rdner
    rdner committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    6fb641d View commit details
    Browse the repository at this point in the history

  5. Rollback unnecessary changes in tests

    @rdner
    rdner committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    62a7b8c View commit details
    Browse the repository at this point in the history

  6. Test fixes

    @rdner
    rdner committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    3769af9 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2023

  1. More tests and docs

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    d8ecc4d View commit details
    Browse the repository at this point in the history

  2. Fix a typo in docs

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    54ae1cd View commit details
    Browse the repository at this point in the history

  3. Apply doc suggestions from code review 

    Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>
    @rdner @kilfoyle
    rdner and kilfoyle authored Jul 13, 2023
    Configuration menu
    Copy the full SHA
    8313958 View commit details
    Browse the repository at this point in the history

  4. Make change to the source of the doc

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    9e6bf34 View commit details
    Browse the repository at this point in the history

  5. Add test for a fingerprint collision in watcher

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    d49fb77 View commit details
    Browse the repository at this point in the history

  6. Promote log messages to the warn level

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    08d8c8e View commit details
    Browse the repository at this point in the history

  7. Use file units in log messages

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    627cef4 View commit details
    Browse the repository at this point in the history

  8. Fix tests

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    e0e8db9 View commit details
    Browse the repository at this point in the history

  9. Handle edge-case when file info is out of date

    @rdner
    rdner committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    11ff189 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2023

  1. Merge branch 'main' into fingerprint-file-identity

    @rdner
    rdner authored Jul 14, 2023
    Configuration menu
    Copy the full SHA
    06cbe45 View commit details
    Browse the repository at this point in the history