Skip to content
This repository has been archived by the owner on Nov 5, 2022. It is now read-only.

Home

Jump to bottom
Glenn K. Lockwood edited this page Sep 26, 2017 · 14 revisions

pytokio

Getting Started

The examples subdirectory contains Jupyter notebooks that demonstrate the basics of using the pytokio API to retrieve data. Because pytokio uses existing data sources, these notebooks must be run on a system that has access to the data source(s) you want. Specifically,

  • NERSC's Lustre data stored in HDF5 is saved to the NERSC Global File System, so accessing it requires access to an NGF mount (e.g., from https://jupyter.nersc.gov/)
  • The job accounting and diameter data relies on Slurm accounting database access via the sacct command. Thus, it must be run from a system that can run sacct -j 12345 and get the correct job information for job 12345.

The tests/ directory also contains a suite of self-contained functionality tests as well as all input data required to exercise them. You can look at those tests to see the bare minimum code necessary to exercise different parts of pytokio, and how you can utilize local data caches to use pytokio on systems that do not satisfy the aforementioned external dependencies.

Contributing

Please don't commit directly to master. Instead, follow the protocol outlined below.

Major Features

All major features should

  1. Have a GitHub issue opened describing the new feature
  2. Have a new branch named according to yourname/githubissuenumber; for example, a branch called glennklockwood/issue23 will correspond to the feature described in Issue #23.

and all commits should be pushed to this branch.

When the feature is complete, it must:

  1. have a test suite in the tests/ subdirectory. New tokio.connectors, tokio.tools, and CLI tools in bin/ must be represented.
  2. demonstrate that all unit/integration tests are passing. This includes running tests that should not have been affected by your changes.

Any new example notebooks in examples/ should be tested to the best of your ability, but there are currently no hard requirements to build unit tests for them. This is subject to change in the future.

Once this has all happened, create a pull request against master for that branch. A second person should review your changes and accept the merge or provide feedback on what must be changed before the branch can be merged. After the branch is merged, it can be deleted.

Bug Fixes

Bug fixes which either

  1. require more than just a line or two of code changes, or
  2. already has a bug filed as a GitHub issue

must follow the same guidelines as the Major Features changes above. Create a new branch named yourname/githubissuenumber, fix the bug there, and then issue a pull request to master when it is complete.

VERY Minor Fixes

Typos, one-line bug fixes, and similarly minor features can be pushed directly to master. Please exercise good judgment when doing this; if your push to master introduces a new bug or breaks a test, you will be in trouble.

Clone this wiki locally