Updating from older version that have written time.Duration causes bad data - maybe add a note?

[flaterik]

We had been using an older version of the driver in production for some time, with many time.Duration values stored in the old nanos format. When we upgraded, all of those durations obviously came back 1e6 times larger than intended. Fixing the data was not difficult once we figured out what was wrong, and I do not foresee a code fix to this, but it might be nice to have a note about this change in the readme! It might save someone else the excitement that I had today.

[domodwyer]

Hi @flaterik

This is a little worrying - to my knowledge we haven't changed this serialisation - do you know which versions are the "old" and "new" in the above so we can put together a test?

Thanks,
Dom

[flaterik]

Sorry, I definitely could have provided more info!
We moved from
package: gopkg.in/mgo.v2
version: 3f83fa5
(which I realize is quite old!)
to
package: github.com/globalsign/mgo
version: 113d396
This is the change that affected things -
go-mgo#373
And this is the code in question -

mgo/bson/encode.go

Line 439 in 113d396

e.addInt64(int64(v.Int() / 1e6))

mgo/bson/decode.go

Line 752 in 113d396

in = time.Duration(time.Duration(d.readInt64()) * time.Millisecond)

In the older incarnations that case wasn't there, so the durations were stored as the underlying time.Duration int64 as nanoseconds.
My solution was to scan our records and divide anything bigger than 1e15 by 1e6, which worked fine. Just had a period of time where things that were supposed to be five minutes were treated as if they should be 83,333.3333 hours.

[domodwyer]

Hi @flaterik

I had forgotten about this! I think it was pulled into my fork before it became a globalsign fork, and once it started becoming popular we decided to not make breaking changes to help users transition - this slipped through!

Thanks for the issue, we'll update the docs!

Dom

[aitva]

Hi @domodwyer,

I am trying to migrate from gopkg.in/mgo.v2 and I have the same issue. Is there a recommended way to fix this? Can you point me at the doc?

Thanks!

[domodwyer]

Hi @aitva

Sorry about the hassle - we have tried pretty hard to maintain compatibility and this was the only slip!

@flaterik seems to have had good results with the strategy above:

My solution was to scan our records and divide anything bigger than 1e15 by 1e6, which worked fine.

But if that is difficult, an alternative is to include a conversion in the application but I would prefer the "change everything once" approach of @flaterik.

Dom