WorkGeneration

Work generators

Submitting one job

Here's a program that submits one job (error-checking is omitted for clarity):

#include "boinc_db.h"
#include "backend_lib.h"

int main() {
    DB_APP app;
    DB_WORKUNIT wu;
    char* wu_template;
    const char* infiles[] = {"infile"};
    char path[1024];

    SCHED_CONFIG config;
    config.parse_file();

    boinc_db.open(config.db_name, config.db_host, config.db_user, config.db_passwd);
    app.lookup("where name='myappname'");

    // write input file in the download directory
    //
    config.download_path("infile", path);
    FILE* f = fopen(path, "w");
    fwrite(f, "random stuff");
    fclose(f);

    read_file_malloc("templates/input_template.xml", wu_template);
    wu.clear();     // zeroes all fields
    strcpy(wu.name, "test_name");
    wu.appid = app.id;
    wu.min_quorum = 2;
    wu.target_nresults = 2;
    wu.max_error_results = 5;
    wu.max_total_results = 5;
    wu.max_success_results = 5;
    wu.rsc_fpops_est = 1e10;
    wu.rsc_fpops_bound = 1e11;
    wu.rsc_memory_bound = 1e8;
    wu.rsc_disk_bound = 1e8;
    wu.delay_bound = 7*86400;
    create_work(
        wu,
        wu_template,
        "templates/output_template.xml",
        "templates/output_template.xml",
        infiles,
        1,
        config
    );
}

The program must be run in the project directory.

Making lots of jobs

If you're making lots of jobs (e.g. to do the various parts of a parallel computation) you'll want the jobs to differ in their input files and/or their command-line arguments.

For example, let's say you want to run a program on ten input files 'file0', 'file1', ..., 'file9'. You might modify the above program with the following code:

    char filename[256];
    char* infiles[1];
    infiles[0] = filename;
    ...
    for (i=0; i<10; i++) {
        sprintf(wu.name, "wu_appName_%d", i);
        sprintf(filename, "file%d", i);
        create_work(
            wu,
            wu_template,
            "templates/output_template.xml",
            "templates/output_template.xml",
            infiles,
            1,
            config
        );
    }

Note that you only need one workunit template file and one result template file.

Now suppose you want to run a program against a single input file, but with ten command lines, '-flag 0', '-flag 1', ..., '-flag 9'. You might modify the above program with the following code:

    char command_line[256];
    ...
    for (i=0; i<10; i++) {
        sprintf(wu.name, "wu_appName_%d", i);
        sprintf(command_line, "-flag %d", i);
        create_work(
            wu,
            wu_template,
            "templates/output_template.xml",
            "templates/output_template.xml",
            infiles,
            1,
            config,
            command_line
        );
    }

Again, you only need one input template file and one output template file.

Sample work generators

Two work generator programs are supplied with BOINC.

Sample work generator

Many projects have an essentially infinite supply of work. This can be handled by a 'flow-controlled work generator' that tries to maintain a constant number of unsent jobs (typically a few hundred or thousand). It does this by periodically querying the BOINC database to find the number of unsent jobs. An example of this is sched/sample_work_generator.cpp. You can easily modify this program to suit your application.

In order to scan a directory for input files and create jobs therefrom, the default sample work generator can be adapted with a patch provided here.

make_work

During the testing phase of a project, you may want to copy a given workunit as needed to maintain a constant supply of work. The daemon program

make_work --wu_name name [--wu_name name2 ... ] --cushion N

does this; it creates copies of the given workunits (cyclically) as needed to maintain a supply of at least N unsent results. Other command-line arguments:

--max_wus N

exit after creating N workunits

--one_pass

check if workunits are needed, if so create them, and exit.

Note: if you run the file_deleter and/or db_purge, the master workunit or its input files may be deleted (which will cause make_work to fail). To avoid this, give the master workunit a name that contains 'nodelete' as a substring. This causes the file_deleter and db_purge to skip over it.

It may be convenient to have a script that recreates the master workunit. For example:

cp test_workunits/12ja04aa `bin/dir_hier_path 12ja04aa`
bin/create_work -appname setiathome -wu_name sah_nodelete -wu_template templates/setiathome_wu_0 -result_template templates/setiathome_result_0 12ja04aa