Skip to content

Longbow v1.4.0

Compare
Choose a tag to compare
@jimboid jimboid released this 07 Jul 14:51
· 215 commits to master since this release

This release is a rather large release with a lot of changes to the code
and also to the way things are being done.

This project is moving towards using CI tools more and more, and in this
release and all future releases tools for automated unit testing and
code quality checks will be used. In this release the following changes
have been made in this regard:

  1. Travis-CI is now used to automate testing for all versions of python
    2.6 through 3.5 https://travis-ci.org/HECBioSim/Longbow

  2. Unit testing added to repository and increased to 100% code coverage.
    Badges have also been implemented so users can monitor success rate.
    Codecoverage is picked up from travis and compiled at
    https://coveralls.io/github/HECBioSim/Longbow.

  3. Automated running of code quality checks is now triggered via github
    pushes and is done at https://landscape.io/github/HECBioSim/Longbow

The actual changes to the code are:

  1. Added in some extra commonly found naming for GROMACS executables
    (gmx_mpi and gmx_mpi_d).

  2. Bugs detected in shellwrappers.py during unit test writing have now
    been fixed.

  3. Fixed formatting issue with logging in staging.py -
    stage_downstream()

  4. Removed import statements for relative imports, this eliminated the
    need for the try/except imports at the top of each code module.

  5. The parameter "frequency" has been split into two parameters
    "polling-frequency" and "staging-frequency". This enables the user to
    have Longbow poll jobs without staging all the time.

  6. The timing mechanism for timing between polling events has now been
    changed from a disruptive wait() to a timestamp comparison. This stops
    the process being "blocked" by wait() and in future will allow the
    addition of other features that can happen on different timelines to
    polling.

  7. There are now two steps to job completion, instead of marking a job
    as finished when it is finished on the remote host and results
    downloaded, Longbow will now mark at as finished as it is finished on
    the remote host and then once staging has happened it will then be
    marked as complete.

  8. Fixed glitch in applications.py processjobs() where the wrong
    comparitor was used on the if statement to construct the upload include
    list

  9. Huge refactoring of code to break down larger methods into smaller
    easier to test methods. All new methods are private methods (starting
    with "_") these should not be used by people making use of Longbow in
    their own code, unless they know what they are doing!

  10. Moved all code for Longbow entrypoints out of the executable and
    into corelibs/entrypoints.py, this allows more options for integration
    and also simplifies it somewhat since the library top level can now be
    imported from the library (otherwise hooking against the executable or
    copying the code into a project was the only way).

  11. The plugin framework no longer has a complicated path such as
    plugins/apps/gromacs.py, now two directories "apps" and "schedulers" sit
    at the same level as corelibs in the library. This removed most of the
    complexity in the import system, now it is a very simple and elegant way
    to provide plugins for these two categories.

  12. Fixed the problem of returning information upon job submission
    error, this was a typo in each of the scheduler plugins.

  13. Fix for strange job status glitch when jobid appeard in say a
    timestamp or some other parameter in the output of qstat etc, this has
    been fixed across all schedulers.

  14. Fixed spacing problem in some generated job submit files.

  15. Modification into the way command-lines are parsed, this is so that
    the detection of executables and their commands as well as Longbow
    commands is much more robust and can now handle arbitrary executables.

  16. Addition of --nochecks command-line flag, this will disable testing
    whether the application is available on the remote host. In some
    circumstances it is very difficult to get Longbow to recognise an
    application is installed. This is aimed at advanced users only.

  17. all parameters in the main entry point that were previous passed
    into longbow have now all been assimilated into one dictionary
    "parameters" this then allowed refactoring all the switch cases for
    command-line parameters into a much neater single method.

  18. Fix for problem detected during unit test writing for
    configurations.py saveconfigs() where if used incorrectly would blow up.

  19. Fixed problem in applications.py causing failed run under python 3.2

  20. Fix for filenaming glitch when using global files in replicates.

  21. Fixed problem where if required files were not found this was
    ignored but should have been flagged up.

  22. Fixed a number of bad initialisers that would cause filenames to go
    missing.

  23. legacy code in status method in all schedulers removed.

  24. Fixed freezing glitch when all jobs failed to submit.

  25. Fixed bad parameter in substitutions.