Home

NEXT is a system that allows crowd-sourcing active learning. We cover some of the features NEXT provides in the Introduction section below, then start a dive into development docs.

Talks give a good brief introduction to NEXT at the highest level. For scientists and develoeprs, we most recommend the PyData Ann Arbor talk. It's an enhanced and refined version of the SciPy talk.

Venue	Audience	Length	Link
PyData Ann Arbor	Scientists and developers	1 hour	https://www.youtube.com/watch?v=rTyu4QTXZTc
SciPy 2017	Scientific Python developers	30 minutes	https://www.youtube.com/watch?v=blPjDYCvppY
Simons Institute conference on Interactive Learning	Machine learning researchers	30 minutes	https://youtu.be/ESXgbZQ1ZTk?t=1732

We give more detail on the items on launching experiments and getting setup in the SciPy 2017 proceedings: http://conference.scipy.org/proceedings/scipy2017/pdfs/scott_sievert.pdf.

After launching NEXT, be sure to visit [next-url]:8000/home which provides links further app argument documentation, the experiment list and dashboards!

• Introduction, which describes some of the motivation and features of NEXT

  • Introduction: an introduction to the NEXT system and how to use it for active learning experiments. This includes links to video that explain NEXT to active machine learning researchers.
  • Starting NEXT, which explains what you see when you launch and where to find documentation.
  • NEXT developer overview including basic terminology, a basic overview of the 5 necessary functions, the built-in apps/interfaces and the folder layout.
  • Experiment Monitoring provides an overview of the tools NEXT provides to help you to monitor experiments in real time.

• Running NEXT. There are several options available to run NEXT available. visit http://[next-url]:8000/home! This provides more documentation, access to the dashboards and system monitors.

  • via local, the recommended method to debug
  • via EC2 AMI (the easiest option but doesn't allow customization)
  • via EC2 with an EC2 account quickstart

• Experiment launch, method to initialize an experiment.

  • Launch Basics which describes the files necessary to launch (and used below).
  • CLI launch. We also provide a detailed launch guide, launching from the command line
  • GUI Launch, launching from the built in GUI
  • Distributing the experiment

• Development basics, the dataflow and debug process

  • NEXT-Basics, which (briefly) describes the different parts of NEXT
  • Basic-Algorithms a description of the 4 key functions needed
  • New-Application. Do you need to develop a new application or interface for your users? Read this first.
  • Detailed data flow: what gets called when with what arguments?
  • Debugging

• Development.

  • NEXT overview that we will use when describing the below points.
  • Locations of a particular app (what files live where?) and a description of each file.
  • Interface to the apps and the Algorithm-Interface.
  • Framework-Apps walks through app development and the necessary functions
  • Framework-Algs walks through algorithm development and the necessary functions
  • Dashboard development, including how to customize and view visualizations
  • Widgets or the user facing HTML
  • Testing, how we validate and time each algorithm
  • Summary
  • Checklist for developing your own app/alg in NEXT.

• Advanced Features

  • Using Redis: butler.memory
  • Diving a bit deeper: Celery workers

• Examples

  • Application-Example
  • Algorithm-Example
  • Butler-Example

• Reference

  • Butler-API
  • Base-Interface
  • Constants
  • Database Layout

• FAQ

• Crowdsourcing Tips

• Glossary