Roadmap

This page captures the goals of the Toil workflow engine project. These are divided into three categories based on when they will be addressed. Given that the project advances thanks to a team of mostly students and external collaborators, these are only rough estimates.

Feedback is welcome. Every effort will be made to put resources on any critical goals.

Near Term (In Progress)

• Basic WDL support.
• Python 3 support.
• Better local testing.
• Anisble provisioning.
• Run Toil within Popper (https://cross.ucsc.edu/tag/popper/).

Medium Term (~ 6-month goals)

• Improve the user experience.
  • Change the thread pool model to improve single machine usage.
  • Better feedback (error messages, logging).
• Update the Azure jobStore.
• Add an Azure provisioner.
• Improve the development process.
  • Switch to Travis.
  • Add a linter
  • Add a code coverage tool.
  • Organize tests.
  • Better access to tests for external developers.
  • Turn off or remove forking to enable easier debugging.
• Better track versions of specifications (e.g. CWL, WDL) and dependencies.

Long Term

• AWS Batch System support
• Google Pipelines support.
• Google Cloud provisioner (Ansible)?
• Singularity support.
• Uniform configuration (i.e. not just environment variables).
• Add management and monitoring UIs.
• TES support
• WES Support

Finished

• Improve the documentation.
• Improve the release process.
• Bring CWL support up to 100%.