Adding levels in CategoricalValue call

[bkamins]

@nalimilan In the following example:

julia> x = categorical([1,2,3])
3-element CategoricalArray{Int64,1,UInt32}:
 1
 2
 3

julia> CategoricalValue(3, x)
CategoricalValue{Int64, UInt32} 3

julia> CategoricalValue(4, x)
ERROR: ArgumentError: level 4 not found in source pool

The question is if maybe we should allow CategoricalValue(4, x) to add level to pool of x (if this would be allowed in assignment to x) rather than erroring?

I am not fully convinced though.

[bkamins]

Alternatively maybe we could allow categorical(scalar) = CategoricalValue(scalar, categorical([scalar])) (+ adding kwarg support I am writing here for brevity)

[nalimilan]

Did this came up in a real use case? Throwing an error has the advantage of being explicit. Otherwise an ordered pool would be made unordered when adding the level, so if you're calling < you'd get an error immediately, but the pool would have been mutated, which can be annoying.

categorical(scalar) is something that we discussed it before IIRC. I guess it could be useful to fix #363 if we allowed x[1] < categorical(scalar) to work for any pool, as there wouldn't be transitivity issues given that categorical(scalar1) < categorical(scalar2) would not be allowed. Is that the kind of thing you had in mind? EDIT: this would actually break transitivity when comparing categorical(scalar) to values from different pools, except if we remembered the pool the first time a comparison is made so that we throw an error for subsequent comparisons with other pools.

[bkamins]

Did this came up in a real use case?

The use case is in JuliaData/DataFrames.jl#2828 where you now have to write e.g.:

unstack(df, :variable, :value, fillvalue=CategoricalValue(0, categorical([0])))

to keep the unstacked columns categorical.

The question is if maybe we should allow CategoricalValue(4, x) to add level to pool of x

After thinking about it I am not a fan of this

categorical(scalar) is something that we discussed it before IIRC

Yes, and there is no rush to decide what we should do as these things are tricky. My intention was just to have a shorter version of CategoricalValue(scalar, categorical([scalar])) which is verbose and non-obvious.

[nalimilan]

Another situation where it could be useful is e.g. in [v == "a" ? categorical("b") : v for v in cat_array], to ensure that the result is categorical.