Add unmet prerequisite dumping on stall.

[arjclark]

Close #1585. Addresses this comment: #1585 (comment)

On stalling the suite will now print out a list of unmet prerequisites for waiting tasks. Where those prerequisites are transitive they will be pruned down as much as possible. i.e. for:

foo => bar => baz

in the event of a stall due to failure of foo only bar will report unmet prerequisites. baz won't as it is waiting on the completion of a waiting task (bar) that hasn't got all its prereqs met.

See the test for the slightly more awkward problem when a task proxy doesn't exist at a cycle, thus resulting in an unmet prerequisite which gets reported (not massively different from what you'd end up with from manually inspecting the task in gcylc).

@hjoliver - please review 1
@matthewrmshin - please review 2

[matthewrmshin]

You can just do for item in prereq_tree:. There is no need to call the .keys() methods.

[arjclark]

@matthewrmshin - tweaked as requested.

[matthewrmshin]

if unsatisfied_id in prereq_tree:

[matthewrmshin]

Some more comments. Otherwise looks OK. Tests OK in my environment.

[arjclark]

@matthewrmshin - addressed.

[hjoliver]

(this will close #1585, right?)

[arjclark]

@hjoliver - that is my hope. Just with you for review now.

[hjoliver]

Pruning doesn't seem to work downstream of an unmet family:

[scheduling]
    [[dependencies]]
        graph = """foo => FAM
            FAM:succeed-any => bar"""
[runtime]
    [[foo]]
        script = false
    [[FAM]]
    [[f_1, f_2, f_3]]
        inherit = FAM

2016-06-20T08:33:02Z CRITICAL - [foo.1] -(current:running)> failed at 2016-06-20T08:33:01Z
2016-06-20T08:33:04Z WARNING - suite stalled
2016-06-20T08:33:04Z WARNING - Unmet prerequisites for f_1.1:
2016-06-20T08:33:04Z WARNING -  * foo.1 succeeded
2016-06-20T08:33:04Z WARNING - Unmet prerequisites for f_3.1:
2016-06-20T08:33:04Z WARNING -  * foo.1 succeeded
2016-06-20T08:33:04Z WARNING - Unmet prerequisites for bar.1:  <------------!!!!!!!!!!!!!!!!
2016-06-20T08:33:04Z WARNING -  *     LABEL: f_3_colon_succeed = f_3.1 succeeded
2016-06-20T08:33:04Z WARNING -  *     LABEL: f_2_colon_succeed = f_2.1 succeeded
2016-06-20T08:33:04Z WARNING -  *     LABEL: f_1_colon_succeed = f_1.1 succeeded
2016-06-20T08:33:04Z WARNING -  * CONDITION: (f_1_colon_succeed | f_2_colon_succeed | f_3_colon_succeed)
2016-06-20T08:33:04Z WARNING - Unmet prerequisites for f_2.1:
2016-06-20T08:33:04Z WARNING -  * foo.1 succeeded

[arjclark]

@hjoliver - fixed (I think).

[arjclark]

(also added a test to capture that kind of prereq filtering)

[hjoliver]

Review 2 - good, tests pass here.