Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON serialization of CategoricalValue{T, R} is broken #95

Closed
alyst opened this issue Oct 30, 2017 · 0 comments · Fixed by #96
Closed

JSON serialization of CategoricalValue{T, R} is broken #95

alyst opened this issue Oct 30, 2017 · 0 comments · Fixed by #96

Comments

@alyst
Copy link
Contributor

alyst commented Oct 30, 2017

Now CategoricalValue is not a <: AbstractString, so JSON.jl serializes it using CompositeTypeWrapper and goes into recursion due to circular pool -> value -> pool references.

julia> using CategoricalArrays, JSON

julia> x = categorical([1, 2, 3]);

julia> JSON.json(x[1])
ERROR: StackOverflowError:
Stacktrace:
 [1] Type at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:18 [inlined]
 [2] lower(::CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:40
 [3] show_pair(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Symbol, ::CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:236
 [4] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294
 [5] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Array{CategoricalArrays.CategoricalValue{Int64,UInt32},1}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:302
 [6] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294
 [7] show_pair(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Symbol, ::CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:236
 [8] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294
 [9] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Array{CategoricalArrays.CategoricalValue{Int64,UInt32},1}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:302
 [10] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294
 [11] show_pair(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Symbol, ::CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:236
 [12] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294
 [13] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Array{CategoricalArrays.CategoricalValue{Int64,UInt32},1}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:302
 [14] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294
 [15] show_pair(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::Symbol, ::CategoricalArrays.CategoricalPool{Int64,UInt32,CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:236
 [16] show_json(::JSON.Writer.CompactContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::JSON.Serializations.StandardSerialization, ::JSON.Writer.CompositeTypeWrapper{CategoricalArrays.CategoricalValue{Int64,UInt32}}) at /home/astukalov/.julia/v0.6/JSON/src/Writer.jl:294

It could be easily fixed by implementing JSON.lower(x::CategoricalValue) method, but that would require introducing CategoricalArrays -> JSON dependency.
The more generic fix would be to implement support for circular references in JSON, but that would still print CategoricalValue incorrectly (probably we just need to show the stored value, i.e. get(x::CategoricalValue)).

alyst added a commit to alyst/CategoricalArrays.jl that referenced this issue Oct 30, 2017
alyst added a commit to alyst/CategoricalArrays.jl that referenced this issue Jan 8, 2018
alyst added a commit to alyst/CategoricalArrays.jl that referenced this issue Jan 8, 2018
@alyst alyst closed this as completed in #96 Jan 16, 2018
alyst added a commit that referenced this issue Jan 16, 2018
fixes #95 

Introduces "hard" JSON dependency, which should be made optional once Julia provides the appropriate mechanisms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant