Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use SecureRandom for UCPListener TCP port choice #2810

Merged
merged 2 commits into from
Jun 25, 2021

Conversation

abellina
Copy link
Collaborator

Signed-off-by: Alessandro Bellina abellina@nvidia.com

Please note that once UCX 1.11 is released, we will remove the need for this choice since UCX will delegate to the kernel to set the ephemeral port.

Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
@abellina abellina requested a review from jlowe June 24, 2021 19:44
@sameerz sameerz added the task Work required that improves the product but is not user facing label Jun 24, 2021
@abellina
Copy link
Collaborator Author

Side note: I did check in this scenario that nextInt reads from /dev/urandom and does not block in any of my tests in Ubuntu or CentOS.

With the default algorithm NativePRNG, nextInt ends up calling implNextBytes which ends up getting random values from /dev/urandom. If you were to call nextBytes, then /dev/random is picked and can block, unless you used NativePRNG.NonBlocking. I did not want to hard specify an algorithm.

@jlowe jlowe added the shuffle things that impact the shuffle plugin label Jun 24, 2021
jlowe
jlowe previously approved these changes Jun 24, 2021
Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit but otherwise lgtm.

@@ -183,6 +183,7 @@ class UCX(transport: UCXShuffleTransport, executor: BlockManagerId, rapidsConf:
worker = context.newWorker(workerParams)
logInfo(s"UCX Worker created")
if (rapidsConf.shuffleUcxUseSockaddr) {
val secureRandom = new SecureRandom()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Why not move this within the one block that needs it? If the port is specified in the conf then this will still construct an unused instance.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

@abellina
Copy link
Collaborator Author

build

@abellina abellina merged commit d008a15 into NVIDIA:branch-21.08 Jun 25, 2021
@abellina abellina deleted the shuffle/use_secure_random branch June 25, 2021 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
shuffle things that impact the shuffle plugin task Work required that improves the product but is not user facing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants