[Feature]: Meta OrdinalClassifier estimator
#611
Conversation
* unit tests * docs * change calibration api
| Examples of this kind of problem are: predicting customer satisfaction on a scale from 1 to 5, predicting the severity of a disease, predicting the quality of a product, etc. | ||
|
|
||
| The [`OrdinalClassifier`][ordinal-classifier-api] is a meta-model that can be used to transform any classifier into an ordinal classifier by fitting N-1 binary classifiers, each handling a specific class boundary, namely: $P(y <= 1), P(y <= 2), ..., P(y <= N-1)$. | ||
|
|
There was a problem hiding this comment.
Bit of a nit, but might it make sense to add this diagram ...
... somewhere over here? ...
There was a problem hiding this comment.
Makes 110% sense! Collapsed admonition should do the trick
| def score(self, X, y): | ||
| """Returns the accuracy score on the given test data and labels. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| X : array-like of shape (n_samples, n_features ) | ||
| The training data. | ||
| y : array-like of shape (n_samples,) | ||
| The target values. | ||
|
|
||
| Returns | ||
| ------- | ||
| score : float | ||
| Accuracy score of self.predict(X) wrt. y. | ||
| """ | ||
| return accuracy_score(y, self.predict(X)) |
There was a problem hiding this comment.
Doesn't the ClassifierMixin provide this already? If we want to do something custom here, perhaps it's better to re-use the score() method of the underlying classifier?
There was a problem hiding this comment.
Oh you are right! I am removing it for now 😊
Not sure what something custom and default would be.
I am personally not paying much attention to accuracy in the ordinal problem I am dealing with, but don't want to put anything very opinionated, so let's go with the hierarchical implementation from ClassifierMixin
| self.classes_ = np.sort(np.unique(y)) | ||
| self.n_features_in_ = X.shape[1] | ||
|
|
||
| if self.n_classes_ < 2: |
There was a problem hiding this comment.
I wonder ... don't we need at least three classes for ordinal regression to make sense? Any binary classification with two classes is automatically ordinal, no?
There was a problem hiding this comment.
Also very true! I cheated for the scikit-learn tests 😂
Let me figure out how to deal with those
| "check_dict_unchanged", | ||
| "check_fit2d_1feature", | ||
| "check_classifier_data_not_an_array", | ||
| "check_classifiers_classes", | ||
| "check_classifiers_train", |
There was a problem hiding this comment.
This is a fair amount of skipping, yet:
- coverage is still same
- the internal estimator will most likely deal directly with these cases (e.g. data not array)
FBruzzesi
changed the title
OrdinalClassifierOrdinalClassifier estimator
Description
Fix #607 by introducing
OrdinalClassifier(meta) estimator.Checklist