[Question] Passing categorical feature with data-type "Category" without passing "categorical_feature"

[mohammad-saber]

Thank you for sharing your great work. I have a question about handling categorical features without using ohe-hot enconding.

Assume that in the dataset, I have 2 categorical features.

df = pd.DataFrame({
    'A': [1, 2, 3, 4, 5],
    'B': ['a', 'b', 'b', 'c', 'c'],
    'C': ['x', 'x', 'y', 'y', 'z'] })

I convert them into ordinal integer values and then convert data-type into "category".

columns_cat = ['B', 'C']

from sklearn.preprocessing import OrdinalEncoder
encoder = OrdinalEncoder(dtype='int')
x[columns_cat] = encoder.fit_transform(x[columns_cat])

for column in columns_cat:
    x[column] = x[column].astype('category')

Question:
When data-type is "Category", do I need to pass parameter categorical_feature when fitting model?

CASE 1:
In the following case, does LightGBM handles columns ['B', 'C'] as categorical?

from lightgbm import LGBMRegressor
model = LGBMRegressor()
model.fit(x, y)  

CASE 2:
And what is the difference when I pass categorical_feature as below:

from lightgbm import LGBMRegressor
model = LGBMRegressor()
model.fit(x, y, categorical_feature=columns_cat)  

Thank you for your time.

[StrikerRUS]

@mohammad-saber Thanks for using LightGBM!

'category' columns in pandas.DataFrame are treated as categorical features by default in LightGBM. So,

When data-type is "Category", do I need to pass parameter categorical_feature when fitting model?

you don't need to pass categorical_feature param in this case.

CASE 1:
In the following case, does LightGBM handles columns ['B', 'C'] as categorical?

Yes, it does.

CASE 2:
And what is the difference when I pass categorical_feature as below:

Should be no difference with CASE 1.

Maybe the following unit test will help to better understand handling categorical features in pandas.DataFrame.

LightGBM/tests/python_package_test/test_sklearn.py

Line 470 in d05f547

def test_pandas_categorical():

Also, please note that ordered categorical columns (pd.Categorical(..., ordered=True)) aren't treated as categorical features by default.

LightGBM/tests/python_package_test/test_sklearn.py

Lines 484 to 485 in d05f547

	"E": pd.Categorical(np.random.permutation(['z', 'y'] * 30),
	ordered=True)})

LightGBM/tests/python_package_test/test_sklearn.py

Lines 515 to 516 in d05f547

	with pytest.raises(AssertionError):
	np.testing.assert_allclose(pred0, pred5) # ordered cat features aren't treated as cat features by default

[mohammad-saber]

Thank you.

Is there any attribute that we can get "features which were treated as categorical" after model fitting?

[StrikerRUS]

You can get this info from the LightGBM logs:

UserWarning: categorical_feature in Dataset is overridden.
New categorical_feature is <your cat features here>

#3379.

Also, categorical features are written differently in a model file.

LightGBM/src/boosting/gbdt_model_text.cpp

Lines 60 to 67 in 48257d4

	} else if (strs[0] != "none") { // categorical feature
	auto vals = CommonC::StringToArray<int>(feature_infos_[i], ':');
	auto max_idx = ArrayArgs<int>::ArgMax(vals);
	auto min_idx = ArrayArgs<int>::ArgMin(vals);
	json_str_buf << "{\"min_value\":" << vals[min_idx] << ",";
	json_str_buf << "\"max_value\":" << vals[max_idx] << ",";
	json_str_buf << "\"values\":[" << CommonC::Join(vals, ",") << "]}";
	} else { // unused feature

[no-response]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.