Alternative to #599 with less logging

[ablaom]

Continuation of #599, implementing variation suggested at #599 (comment)

julia> e = evaluate(model, X, y, measures=[LogLoss(), Accuracy()])
Evaluating over 6 folds: 100%[=========================] Time: 0:00:00
PerformanceEvaluation object with these fields:
  measure, measurement, operation, per_fold,
  per_observation, fitted_params_per_fold,
  report_per_fold, train_test_pairs
extract:
┌─────────────────────────────────┬─────────────┬──────────────┬──────────────────────────────────────────────────┐
│ measure                         │ measurement │ operation    │ per_fold                                         │
├─────────────────────────────────┼─────────────┼──────────────┼──────────────────────────────────────────────────┤
│ LogLoss(tol = 2.22045e-16) @362 │ 1.08        │ predict      │ [2.22e-16, 2.22e-16, 2.12, 2.12, 2.25, 2.22e-16] │
│ Accuracy() @097                 │ 0.97        │ predict_mode │ [1.0, 1.0, 0.941, 0.941, 0.938, 1.0]             │
└─────────────────────────────────┴─────────────┴──────────────┴──────────────────────────────────────────────────┘

[codecov-commenter]

Codecov Report

Merging #600 (a218207) into dev (d0726dd) will increase coverage by 0.56%.
The diff coverage is 84.50%.

@@            Coverage Diff             @@
##              dev     #600      +/-   ##
==========================================
+ Coverage   85.15%   85.71%   +0.56%     
==========================================
  Files          37       37              
  Lines        3192     3269      +77     
==========================================
+ Hits         2718     2802      +84     
+ Misses        474      467       -7     

Impacted Files	Coverage Δ
src/MLJBase.jl	100.00% <ø> (ø)
src/operations.jl	80.76% <ø> (ø)
src/show.jl	36.64% <50.00%> (+15.12%)	⬆️
src/resampling.jl	89.79% <87.30%> (+0.25%)	⬆️
src/measures/measures.jl	94.00% <100.00%> (+3.52%)	⬆️
src/composition/abstract_types.jl	40.00% <0.00%> (-21.54%)	⬇️
src/measures/finite.jl	92.71% <0.00%> (-3.89%)	⬇️
src/univariate_finite/arrays.jl	100.00% <0.00%> (ø)
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0726dd...a218207. Read the comment docs.

[ablaom]

@CameronBieganek Assuming you're pretty busy. But in case this fell off your radar... 🙏🏾

[CameronBieganek]

@ablaom Yeah, I've been pretty tied up lately. And I'm going on a short trip Sunday through Wednesday. But I would definitely like to help—I'll try to squeeze this in before I leave. 😃

[CameronBieganek]

Looks pretty good overall. I made a few minor comments here and there.

Are all those verbosity=0 in the unit tests still needed with the new version of this PR?

[CameronBieganek]

Maybe you can reduce the nesting of the if blocks by writing

if (model isa Probabilistic) && (operation == predict) && (prediction_type(measure) != :probabilistic)

I'm guessing you wrote it your way for readability or to stay within the line length limit, but I think the logic is easier to understand when the conditions are combined into one boolean expression.

Maybe something like this for a readable typesetting:

is_non_probabilistic_measure = (
    model isa Probabilistic &&
    operation == predict &&
    prediction_type(measure) != :probabilistic
)

if is_non_probabilistic_measure
    # ...

[ablaom]

Good suggestion. Adopted.

[ablaom]

@CameronBieganek Many thanks for looking over this!

Sorry about verbosity=0 solution. I should have put that in a separate PR. No it was not needed here. Just that all the noise in the test log was driving me batty.