API versioning

[redboltz]

Overview

@nobu-k and I are considering API versioning on the C++ part of the msgpack-c. Let's say the next release is v1. We can call msgpack APIs as follows:

msgpack::foo();     // call v1::foo()
msgpack::v1::foo(); // call v1::foo() explicitly

When the new version of the msgpack-c API, v2, is released, msgpack::foo() is dispatched to v2::foo() :

msgpack::foo();     // call v2::foo()
msgpack::v1::foo(); // call v1::foo() explicitly
msgpack::v2::foo(); // call v2::foo() explicitly

To achieve that, we use inline namespace on C++11. See http://www.stroustrup.com/C++11FAQ.html#inline-namespace

For C++03, we use using namespace directive. It is not 100% compatible to inline namespace. For example, overload resolution is different. I refer to overload resolution later.

Versioning header file

We introduce versioning.hpp that is included from all msgpack headers:

// versioning.hpp
#if __cplusplus < 201103

namespace msgpack {
    namespace v1{}
    using namespace v1;
}

#else

namespace msgpack {
    inline namespace v1{}
}
#endif

Adding a new version

After v2 is relased, versioning.hpp is updated as follows:

// versioning.hpp
#if __cplusplus < 201103
namespace msgpack {
    namespace v2{}
    using namespace v2;
}

#else

namespace msgpack {
    inline namespace v2{}
}

#endif

Client code doesn't need to change. If the v2 library code introduce a new version of the function, just add the function into the v2 namespace. If the v2 library code use the v1 functions, use using declarations:

http://melpon.org/wandbox/permlink/PYGoqDAViUcwqF4F

#include <iostream>

// versioning.hpp
#if __cplusplus < 201103

namespace msgpack {
    namespace v2{}
    using namespace v2;
}

#else

namespace msgpack {
    inline namespace v2{}
}

#endif

// lib code v2 (before v1 code). New definitions.
namespace msgpack {
    namespace v2 {
        void foo() {
            std::cout << "v2::foo()" << std::endl;
        }
    }
}

// lib code v1
namespace msgpack {
    namespace v1 {
        void foo() {
            std::cout << "v1::foo()" << std::endl;
        }
        void bar() {
            std::cout << "v1::bar()" << std::endl;
            foo();              // A. v1::foo()  same as defining namespace
            msgpack::foo();     // B. v2::foo()  defaulted by inline namespace
            msgpack::v1::foo(); // C. v1::foo()  explicit call
            msgpack::v2::foo(); // D. v2::foo()  explicit call
        }
    }
}

// lib code v2 (after v1 code). Using decralations.
namespace msgpack {
    namespace v2 {
        using v1::bar;
    }
}

// client code
int main() {
    msgpack::bar();
}

To achive this dispatching, we need to write the library code carefully. See bar() in the v1. We have 4 ways to call foo() in bar(). D is not practical because we don't know the v2 at that time. B is a kind of extension point. The namespace of foo() is determind by versioning.hpp. I believe that it is a good choice as a default implementation. If you want to call the specific version API, use C. If you want to call the namespace same as the current namespace, use A.

When the library code is implemented as B approach, we need to provide the using declarations for callee funtions as follows:

http://melpon.org/wandbox/permlink/RHytaexZBR1pqCt9

#include <iostream>

// versioning.hpp
#if __cplusplus < 201103

namespace msgpack {
    namespace v2{}
    using namespace v2;
}

#else

namespace msgpack {
    inline namespace v2{}
}

#endif

// lib code v2 (before v1 code). Using decralations.
namespace msgpack {
    namespace v1 {
        void foo(); // v1's function declaration
    }
    namespace v2 {
        using v1::foo;
    }
}

// lib code v1
namespace msgpack {
    namespace v1 {
        void foo() {
            std::cout << "v1::foo()" << std::endl;
        }
        void bar() {
            std::cout << "v1::bar()" << std::endl;
            foo();              // A. v1::foo()  same as defining namespace
            msgpack::foo();     // B. v2::foo()  defaulted by inline namespace
            msgpack::v1::foo(); // C. v1::foo()  explicit call
        }
    }
}

// lib code v2 (after v1 code). Using decralations.
namespace msgpack {
    namespace v2 {
        using v1::bar;
    }
}

// client code
int main() {
    std::cout << "Call msgpack::bar() from a client." << std::endl;
    msgpack::bar();
    std::cout << "Call msgpack::foo() from a client." << std::endl;
    msgpack::foo();
}

Note: using v1::foo after the v1::bar() definition, bar() can't find it:

http://melpon.org/wandbox/permlink/M2S4WvtnyeoSCKWn

Overload resolution

When we use C++11, we can write overload functions as follows:

// user overload
namespace msgpack {
    inline void operator>>(object const&, int&) {
        std::cout << "OL operator>>(object const& int&)" << std::endl;
    }
}

The default version is v1.

http://melpon.org/wandbox/permlink/2FZMcXYpxawZhZLZ

When we use C++03, we need to write the overload as follows:

// versioning.hpp
#define CURRENT_VERSION v1

// user overload
namespace msgpack {
    namespace CURRENT_VERSION {
        inline void operator>>(object const&, int&) {
            std::cout << "OL operator>>(object const& int&)" << std::endl;
        }
    }
}

http://melpon.org/wandbox/permlink/oHt1W5dJI2BLmY0Y

If the default version is v2, the overloads doen't work well.

http://melpon.org/wandbox/permlink/iM9qSF2lAMLUzTJc

The line 48 on above code dispathes to v1::operator>> :

                *this >> v;                     // always dispatch to v1, work well with user overload

The line 49 dispatches to v2::operator>>, it's good but doesn't work well with user overloads:

                 msgpack::operator>>(*this, v);  // dispatch to inline namespace, not work well with user overload

If we write the namespace v1 explicitly, it's works well but the default version is v2. It's hard to maintain:

// user overload
namespace msgpack {
    namespace v1 {
        inline void operator>>(object const&, int&) {
            std::cout << "OL operator>>(object const& int&)" << std::endl;
        }
    }
}

http://melpon.org/wandbox/permlink/nQRgsybHwDdp2kAv

It's not a practial solution.

Conclusion

Current msgpack-c depends on ADL. It doesn't work well with this versioning mechanism. I almost give up it. Any ideas?

[redboltz]

I found a new solution. See http://melpon.org/wandbox/permlink/gwcSh49nTnHYNJac

#include <iostream>

// versioning.hpp
#define CURRENT_VERSION v2

#if __cplusplus < 201103

namespace msgpack {
    namespace v2{}
    using namespace v2;
}

#else

namespace msgpack {
    inline namespace v2{}
}

#endif

// msgpack_object_pre.hpp
// lib code v2 (before v1 code). New definitions.
namespace msgpack {
    namespace v1 {
        struct object;
    }
    namespace v2 {
        using v1::object;
    }
}


// user overload
// Must define before caller ( in this case convert() )
namespace msgpack {
    namespace CURRENT_VERSION { // If you use only C++11, you can omit this line
        inline void operator>>(object const&, int&) {
            std::cout << "OL operator>>(object const& int&)" << std::endl;
        }
    }
}


// lib code v2 (before v1 code). New definitions.
namespace msgpack {
    namespace v2 {
        template <typename T>
        inline void operator>>(object const&, T&) {
            std::cout << "v2::operator>>(object const& T&)" << std::endl;
        }
    }
}

// lib code v1
namespace msgpack {
    namespace v1 {
        struct object;
        template <typename T>
        inline void operator>>(object const&, T&) {
            std::cout << "v1::operator>>(object const& T&)" << std::endl;
        }
        struct object {
            template <typename T>
            void convert(T& v) const {
                // *this >> v;                  // no longer depends on ADL
                msgpack::operator>>(*this, v);  // dispatch to inline namespace, work well with user overload defined above
            }
        };
    }
}

// client code

int main() {
    msgpack::object o;
    int i = 1;
    double d = 1.0;
    o.convert(i);
    o.convert(d);
}

In this approach, msgpac-c library code no logner depends on ADL.

Line 66:

                msgpack::operator>>(*this, v);  // dispatch to inline namespace, work well with user overload defined above

User overload should place before caller function's definition. So, msgpack-c need to provide the new header file that includes minimum definition and declaration for overloads.

User overload functions are located between msgpack_object_pre.hpp and msgpack.hpp as follows:

#include <msgpack_object_pre.hpp>

// user overload
// Must define before caller ( in this case convert() )
namespace msgpack {
    namespace CURRENT_VERSION { // If you use only C++11, you can omit this line
        inline void operator>>(object const&, int&) {
            std::cout << "OL operator>>(object const& int&)" << std::endl;
        }
    }
}

#include <msgpack.hpp>

The header file name is not considered well yet.

Is that a good solution?

[redboltz]

I got warnings in the following code:
http://melpon.org/wandbox/permlink/vTgFyPUAmsRGHgPD

And I found it:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53402