Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checking model instances are not types #782

Open
ablaom opened this issue Jun 9, 2022 · 1 comment
Open

Checking model instances are not types #782

ablaom opened this issue Jun 9, 2022 · 1 comment
Labels
good first issue Good for newcomers

Comments

@ablaom
Copy link
Member

ablaom commented Jun 9, 2022

A very common gotcha for users (not necessarily beginners) is calling a method with a model type instead of a model instance. Sometimes the consequence is a confusing error message.

Sometimes we check this (eg, here) but sometimes we don't (eg, base models in a Stack).

In some cases (eg, pipelines) we catch a type and automatically instantiate a default instance, if necessary. I have mixed feelings about this. For example, some model wrappers, eg, TunedModel, EnsembleModel, cannot be called with zero arguments (an atomic model must be specified). So I propose we eventually deprecate this behaviour

For now, I would like a review of MLJBase.jl which introduces the check everywhere a user is expected to call a method with an instance, but might supply a type by accident. An informative error is to be thrown. This could start with some infrastructure along the lines of:

const MSG_MODEL_EXPECTED  = "Type encountered where model instance expected"
function check_is_instance(model, message=MSG_MODEL_EXPECTED)
    model isa Model || throw(AssertionError(message))      # in future, use `ismodel(model)`
    return nothing
end 

cc @rikhuijzer

@ablaom
Copy link
Member Author

ablaom commented Aug 5, 2022

I've made some progress here by adding a check_ismodel method to MLJBase #819 and implementing it for Stack.

Left to do is to implement the check in:

  • TransformedTargetModel (MLJBase.jl)
  • EnsembleModel (MLJEnsembles.jl)
  • learning network machines ? (MLJBase.jl)
  • TunedModel (MLJTuning.jl)
  • IteratedModel (MLJIteration.jl)

(For now pipelines allow types to be used in place of instances, which I think was a bad idea, but changing that now would be breaking.)

@ablaom ablaom added the good first issue Good for newcomers label Jan 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant