Managed Yarn AppMaster & ClusterManager. Credentials and ContainerId implementations. #46
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
implement Credentials with ability to interpret stored tokens
- add method to extract containerid and appid from environment set by yarn - now that we can uderstand the container id that yarn sets up (above), use that instead of serializing via JULIA_YARN_CID - allow multiple args to be specified for launch command as a vector of strings, useful in composing command line for execution
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JULIA_YARN_CIDany more.Running as a managed app master would allow Julia Applications to be run completely inside Yarn, unlike the case of unmanaged app master where the master process hangs outside the cluster. The submitting process can terminate safely, leaving the submited Julia process running inside the cluster. The Julia process running inside the Yarn cluster can then start other worker processes as it desires - all running inside Yarn. Status of the submitted application can be queried anytime, or viewed on the Yarn status board.