Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KPL Binary installation uses a Shared Lock #91

Closed
skidder opened this issue Feb 24, 2017 · 1 comment
Closed

KPL Binary installation uses a Shared Lock #91

skidder opened this issue Feb 24, 2017 · 1 comment
Labels
bug
Milestone
v0.12.4

Comments

@skidder
Copy link
Contributor

skidder commented Feb 24, 2017 •
edited
Loading

The KPL binary is vulnerable to a race condition that yields a zero-byte binary file being installed. The sequence is:

  1. Process-1 determines that the binary does not exist and creates a new FileOutputStream, which will create a file-descriptor on the filesystem (link)
  2. Process-2 determines that the file exists and creates a FileInputStream and shared lock: (link)
  3. Process-1 attempts to create an exclusive lock but cannot because Process-2 has a shared-lock that prevents the creation of exclusive locks: (link)
  4. Process-2 verifies the contents of the binary and throws a SecurityException because the contents do not match the expected value: (link)

This can be avoided by using a dedicated lock-file separate from the binary file being written.

This issue is related to #55
@skidder skidder mentioned this issue Feb 24, 2017
Merged
@pfifer pfifer added the bug label May 3, 2017
@pfifer
Copy link
Contributor

pfifer commented May 3, 2017

Thanks for reporting this, and providing the PR.

@pfifer pfifer added this to the v0.12.4 milestone May 16, 2017
pfifer added a commit that referenced this issue May 17, 2017
@pfifer
Release 0.12.4 of the Amazon Kinesis Producer Library
5fb4008 
=== 0.12.4

==== Java

* Upgraded dependency on aws-java-sdk-core to 1.11.128, and removed version range.
  * [PR #84](#84)
  * [PR #106](#106)
* Use an explicit lock file to manage access to the native KPL binaries.
  * [Issue #91](#91)
  * [PR #92](#92)
* Log reader threads should be shut down when the native process exits.
  * [Issue #93](#93)
  * [PR #94](#94)

==== C++ Core

* Add support for using a thread pool, instead of a thread per request.
  The thread pool model guarantees a fixed number of threads, but have issue catching up if the KPL is overloaded.
  * [PR #100](#100)
* Add log messages, and statistics about sending data to Kinesis.
  * Added flush statistics that record the count of events that trigger flushes of data destined for Kinesis
  * Added a log message that indicates the average time it takes for a PutRecords request to be completed.

      This time is recorded from the when the request is enqueued to when it is completed.
  * Log a warning if the average request time rises above five times the configured flush interval.

      If you see this warning normally it indicates that the KPL is having issues keeping up. The most likely
      cause is to many requests being generated, and you should investigate the flush triggers to determine why flushes
      are being triggered.
  * [PR #102](#102)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
v0.12.4
Development

No branches or pull requests
2 participants
@skidder @pfifer