Referee

What is it?

The first phase of world domination, of course!

No, really.

You're no fun, are you? Fine. If you must know, it's part of a new feedback system for CS courses at St. Olaf.

Contributing

We've got a Python virtual environment set up for this, so that you don't need to muck with your global python installation:

source bin/activate       # bash
source bin/activate.csh   # csh
source bin/activate.fish  # fish

After you're in the venv, make sure to install the requirements we have:

make init

Goal: Provide more immediate feedback to students about their homework submissions. Un-goal: Remove human graders from the feedback loop.

Student submits homework: git commit -m "submit hw13 complete" and git push origin master
Stogit sees a new commit and notifies a webhook: http://rns202-3.cs.stolaf.edu:15715/stogit.py
referee now has control of the process!
- It checks to see if the commit came from a stogit group that we care about
- Then it loads the appropriate course definitions
- It checks the commit message against something, to see if it needs to care about it: something like (hw|lab).*complete$
- If the commit makes it here, we send the student an email confirming that they successfully submitted homework
- Now we load a spec file for the appropriate course:
  - Check to see if folder exists
  - For each file that we expect, check to see if it's there
  - Also check for common misspellings and case-sensitivity
  - Also check for the filename in other folders? Might provide too many false positives
  - Lint each file, with custom syntax-checker, cpplint/cppcheck, and a style checker?
  - Try to compile each file
  - Test files as appropriate
Done! Send student an email with feedback from each successful step and the failed step. Stop processing as soon as an error occurs.