TypeError: non-boolean (Missing) used in boolean context

[scls19fr]

Hello,

I'm reading a GPX file (GPS track) https://gist.github.com/scls19fr/3048506102e37263902588f86b7e759f using Julia.

Here is my code

using Dates
using XMLDict


function read_gpx(fname)
    s = read(fname, String)
    xml = parse_xml(s)
    trkpts = xml["trk"]["trkseg"]["trkpt"]
    g = (
            (
                time=parse(DateTime, trkpt["time"], dateformat"yyyy-mm-dd\THH:MM:SSZ"), 
                elevation=parse(Float64, trkpt["ele"]),
                description=trkpt["desc"],
                latitude=parse(Float64, trkpt[:lat]),
                longitude=parse(Float64, trkpt[:lon])
            )
    for trkpt in trkpts)

    return g
end

fname = "sample.gpx"
# fname = "20191107_LFBI_0932_LFBI_1134.gpx"
g = read_gpx(fname)
positions = collect(g)
# println(positions)

using DataFrames
df = DataFrame(g)
println(df)

using CSV
CSV.write("sample.csv", df)

# using CSVFiles
# save("sample.csv", g)  # raises ERROR: LoadError: TypeError: non-boolean (Missing) used in boolean context

Exporting to CSV file by converting to DataFrame and using CSV.jl works fine but I can't directly export my position generator to CSV file using CSVFiles.jl

It's raising ERROR: LoadError: TypeError: non-boolean (Missing) used in boolean context.

I don't understand what is going on. I wonder if that's bug on CSVFiles or a misunderstanding from me how to use it.

Any idea?

Kind regards

[davidanthoff]

You are trying to save g, but you should try to save df. g is not a valid table in the TableTraits.jl sense.

Having said that, I just discovered a but in the write functionality while trying this out. Fix is in #66, and I'll tag a release ASAP. Without that fix it won't properly quote the string columns.

[scls19fr]

Sorry @davidanthoff but I don't understand why a generator of NamedTuple can't be be directly used for saving to CSV.

I don't think it should be necessary to allocate size for storing a DataFrame for saving to CSV after.

Generator of NamedTuple could (should?) be directly consumed (as DataFrame can consume such generator)

[davidanthoff]

The TableTraits.jl interface requires that you represent missing values using the DataValue type from DataValues.jl, not as Union{T,Missing}. The short version of a very long story behind that is that Union{T,Missing} is not designed to work well in some of the core scenarios that Query.jl supports, so I can't use it as the core missing value datatype in queryverse, whereas DataValue works well. So this implies that the table traits interface mandates that missing values are represented via DataValue. If you pass a generator that does this to the save method, everything will work.

This is obviously not a great situation, but unfortunately just where we are currently with the technical limitations that the Missing story has at this point.

[scls19fr]

ok so I will allocate a DataFrame as it seems to me to be the simplest approach (at least for now and until situation of Missing is at this point)
Thanks @davidanthoff for your help.