From ba2a29745450cd2f6ec1375fbe722ad07ef76200 Mon Sep 17 00:00:00 2001 From: Lance Hepler Date: Sat, 10 Aug 2013 17:12:22 -0700 Subject: [PATCH] doc: update parallel-computing for ClusterManager --- doc/manual/parallel-computing.rst | 35 +++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/doc/manual/parallel-computing.rst b/doc/manual/parallel-computing.rst index 96b01ff9a0adb..27b2573b9f3d9 100644 --- a/doc/manual/parallel-computing.rst +++ b/doc/manual/parallel-computing.rst @@ -589,3 +589,38 @@ required, since the threads are scheduled cooperatively and not preemptively. This means context switches only occur at well-defined points: in this case, when ``remotecall_fetch`` is called. + +ClusterManagers +--------------- + +Julia worker processes can also be spawned on arbitrary machines, +enabling Julia's natural parallelism to function quite transparently +in a cluster environment. The ``ClusterManager`` interface provides a +way to specify a means to launch and manage worker processes, for +example:: + + immutable SSHManager <: ClusterManager + launch::Function + manage::Function + machines::AbstractVector + + SSHManager(; machines=[]) = new(launch_ssh_workers, manage_ssh_workers, machines) + end + + function launch_ssh_workers(cman::SSHManager, np::Integer, config::Dict) + ... + end + + function manage_ssh_workers(id::Integer, config::Dict, op::Symbol) + ... + end + +where ``launch_ssh_workers`` is responsible for instantiating new +Julia processes and ``manage_ssh_workers`` provides a means to manage +those processes, e.g. for sending interrupt signals. New processes can +then be added at runtime using ``addprocs``:: + + addprocs(5, cman=LocalManager()) + +which specifies a number of processes to add and a ``ClusterManager`` to +use for launching those processes.