Test&Score: Instantly recognize new input

[ajdapretnar]

Orange version

3.3.10-dev

Expected behavior

When connecting a new instance of test data to Test&Score, the widget immediately reports 'Test data present, but unused'.

Actual behavior

Test&Score stays silent and doesn't trigger a warning.

Steps to reproduce the behavior

File - Select Rows - connect two inputs to Test&Score. By default the second input is 'Matching Data - Test Data'. Change it to 'Remaining Data - Test Data'. The widget still doesn't trigger a warning. When Test data is selected and then unselected, the warning appears.

[SilentFlame]

@ajdapretnar are we connecting the two inputs to Test&Score from Select Rows only?
or something else is the case you are talking about?

Please let me know, as would like to work on this.

[ajdapretnar]

This is the mentioned workflow. Basically, the warning you see above Test & Score should trigger immediately upon connecting a channel to 'Test Data'.

Now, only when I change from cross validation to any other method does the warning trigger.

[SilentFlame]

@ajdapretnar still unable to understand the problem.

as in my case no warning triggers.

please provide some more insights on the issue.

[ajdapretnar]

The reason for the above issue is that your 'Unmatched Data - Data' has no data as indicated by the dashed line. This probably means your 'Matching Data' subset is also the entire data set. You need to select a subset smaller than the entire data and construct the same workflow.

Now, when I connect the link to 'Test Data' (from either Matching Data or Unmatched, doesn't matter), the warning should trigger to notify the user that he or she should switch to 'Test Data' method. Now it doesn't initially. It's just for greater user friendliness.

To replicate, you need:

1. Train data as Data input.
2. Learner (e.g. LogReg).
3. First the model should be trained with cross validation in T&S.
4. Now connect test data to Test Data input.
5. The warning should trigger, notifying the user that it has test data on the input but is not using it.
6. The warning triggers when I switch to any other method (say Random sampling).

Same warning should be triggered as soon as Test&Score has Test Data on the input.