Open Upgrade Community Manager: A new tool to facilitate any type of contributions

[jcdrubay]

Recently, a Survey was sent to request feedback on OpenUpgrade.
In the past, I have used OpenUpgrade several times, I have also been using custom made migration tools (not OpenUpgrade).

I think that OpenUpgrade is a great solution which deserves way more support than it does.
Also, the number of contributors should be a lot more considering how useful and wonderful that solution is.
The existing contributors are highly knowledgeable, committed and supportive people.

Here are my 2 cents, I hope this could help on how to make contributions to OpenUpgrade more "attractive".

In brief, I believe that the current tools based on github PR, Issues and Discussions in a single repo won't allow scaling because they are too demanding on quality for non-expert contributors. The current tools are forcing contributors to be "perfect" contributors. But too many of us are not as good and motivated as the existing contributors...

I imagine that the audience for this new tool(s) is NOT the current maintainers of OpenUpgrade, but all the developers and companies who got discouraged to do any contributions and just wait for OpenUpgrade to be fully ready in the target version before them using it.

Here is a draft specification of what this new tool could be:
https://docs.google.com/presentation/d/1DRd021SUYC3_urTAn-AanJlSDSL5k2xziIh4nMM6jZY/edit#slide=id.p

[legalsylvain]

Hi.

Thanks @jcdrubay for your interest to increase Openupgrade contributions.

Well, in fact, I don't have the same analysis regarding the situation. Could you elaborate some assertion ?

1. For example this one : "the current tools based on github PR, Issues and Discussions in a single repo won't allow scaling because they are too demanding on quality for non-expert contributors. The current tools are forcing contributors to be "perfect" contributors"

I don't see what you mean. I observe new people arriving who have "various levels" of contributions. I feel like you can contribute even if you're not an Openupgrade expert. Could you provide examples ?

2. Also this one "but all the developers and companies who got discouraged to do any contributions".

I'm not sure who you're talking about.

Note : I'm not saying that there are no worries at the moment, in terms of quantity of contributors, but I'm not sure I agree with the diagnosis.

[jcdrubay]

People who do not contribute are not visible, or not much... so it's difficult to find out. I speak from my own experience and the chats I had with a developers from other companies.

If you check the list of contributors of the past 5 months, you will see that some of them are coming from the same company Viindoo, so this company has decided to invest in OpenUpgrade because they really want to have it within a fixed deadline. Luckily the OpenUpgrade maintainers have been very quick to respond, review all PRs to encourage them to continue.

Other contributors are you, Pedro, Miquel and Stefan. Then there are very few and very minor contributors.

https://github.com/OCA/OpenUpgrade/graphs/contributors

From the search I did for help request since May 1st, I could see that there is no "transformation" of people asking for help and later on contributing anything (I have added one more slide in my presentation).

I myself did some contributions in the past which have been rejected because they did not follow the standard or where workarounds. It's great they were rejected because they were truly not mature. Yet, until a better solution could be merged, these rejected suggestions were the workaround I needed to progress and even complete the upgrade of some projects. I wish these contributes could have helped others.

[tranngocson1996]

I also see that there have been almost no contributions lately other than a few acquaintances and people at our company.

I think there are several reasons that contributions are being reduced like this:

• People who can merge or review are too busy
• Those who need this repo to migrate the db are waiting for the code to be available instead of starting to work. Because when they do it also doesn't seem to get much benefit (in my personal opinion)

I understand what @jcdrubay is trying to say and I think it's a good idea

[rvalyi]

Hello @jcdrubay I did a bit the same remark to @legalsylvain privately 2 days ago. I feel that many Openupgrade users will figure out some kind of dirty SQL hacks or OU patches for their specific database issues but won't be able to contribute proper PRs (lack of expertise or time).

Often migrating a real Odoo instance takes anything like 2x to 10x more than what an average customer would think it would take. So whenever one "saves" a customer from obsolescence, there is hardly enough budget left to contribute back to Openupgrade properly; you already gave all your free time away.
Hopefully this cultural issue improves a bit as companies under Odoo evolve a lot a migration is hard and expensive. Better docs about real life OU contributing could also help a bit.

But all in all, I felt Openupgrade could still take advantage from crowd sourcing all these little dirty hacks people do to get the job done in their database.

Indeed if you have an issue migrating module X, you may save a previous time finding out other people got the same issue and how they worked around it.

And finally the more serious contributors may also like to gather these hacks so they get more chances to convert them into proper Openupgrade scripts.

So I think it's a bit the same diagnosis as you did. Not sure what would be the best solution, may be proper issue labels, may be something else? A StackOverflow section?

Also not all hacks deserve an OU script change. Example: my V8 customer had a customization to get a char field as HTML while Openupgrade would try to convert that char field to HTML again. So I had to disable it in Openupgrade. That was easy for me but may be a blocker for people with less experience. May be my hack could have been in a place people could eventually find it. I'm mean I'm not sure we can really do something about it, but just sharing my experience anyway.

[jcdrubay]

There are 554 forks of OpenUpgrade, how many contains code which could be useful? Out of these 554 forks, there are a lot of potential contributions.

[legalsylvain]

OK.
If I understand correctly one of the question is : "where is the place for "not finished" / "not perfect" / "dirty hack" openupgrade contributions ?"
Regarding raphael valyi options ("may be proper issue labels, may be something else? A StackOverflow section?")
I think that the first Keep It Simple Solution could be:

• add a proper label on issue / PR to identify such contributions.
• add a simple link on the main openupgrade project. Somethink like. "For work in progress and controversial contribution, please search potential solutions here"

People then can create Issue / PR with a title like "11.0 - Stock - dirty hack to handle a use case ..." and we flag the PR.

Then, if there a thousands of contributions, we'll talk about a more complex / adapted tools. At this stage of this project, developing a dedicated tool seems a bit premature (described in the power point document), and energy consuming.

Is it OK for you ?

[jcdrubay]

Thanks for all the feedback.

Due to an overwhelming curiosity, I am not giving up on the initial idea.
I will try to publish an instance online soon and will share access.

As a first step, I am trying to compute some metrics which could facilitate the assessment of the workload when deciding to start an upgrade and hopefully also guide on where to start.

Maybe, the whole part of dirty contributions won't fit into the first version but instead could work hand in hands with the suggestions to use Github with tags

I fully agree that it is a complete overkill for the current community of contributors and number of contributions... but hopefully this will help (at least to satisfy my curiosity in playing with Github APIs, and trying to define a tool could change the future of OCA).

Most likely this attempt will be a failure, but with a hope that it can trigger useful ideas.

I think that using PR or issues would pollute the OpenUpgrade repo, however discussions could be more appropriate.
It could be a new category in discussions and a tag per module would be great.
Without tag, a potential tool could try to guess from the name of the issue/discussion/PR.

[legalsylvain]

OK !
I think I have spoken enough.
Maybe other historical contributors could give their point of view ? @pedrobaeza, @StefanRijnhart, @MiquelRForgeFlow.
kind regards

[jcdrubay]

I have made some progress and I added some screenshots to the presentation: https://docs.google.com/presentation/d/1DRd021SUYC3_urTAn-AanJlSDSL5k2xziIh4nMM6jZY/edit#slide=id.g1349d4c55fd_0_59

I could manage to have 100% of data pulled from Github without any manual entry.

So this project is really layer on top of github which places the "(module,version)" couple at the center while Github places the repository in the center.

The scope is now completely stepping out of OpenUpgrade to enable having an holistic view per module for a specific version.
I imagine that it could help maintainers and users to (automatically) add a link in the README of each module pointing to the website page of this project to provide all info about the module.
Should this discussion be moved to https://github.com/OCA/maintainer-tools ?

I plan to implement several points which are documented in the slides of "Readme extracts"
from the presentation.

Once the Prototype phase is complete, I plan to:

• publish the project so anyone can test as a public user and I will invite some people upon request (let me know if you are interested)
• publish the code of the project

[jcdrubay]

Hello,

I have published a first version.

Here are the links to the few main public pages:

• Overview of the readiness for upgrades targeting a specific Odoo Version based on some "projects" (from the backend, it is possible to add custom projects with a custom list of modules): https://oca-cm-15.komit.link/odoo-version?&version=15.0
• A module with a migration script in a Pull Request: https://oca-cm-15.komit.link/odoo-module-version?repository=odoo%2Fodoo&module=stock_account&version=15.0
• A module with existing migration scripts in a fork of OpenUpgrade without Pull Request https://oca-cm-15.komit.link/odoo-module-version?repository=odoo%2Fodoo&module=account&version=15.0
• A module with a Pull Request to upgrade the code: https://oca-cm-15.komit.link/odoo-module-version?repository=OCA%2Faccount-financial-tools&module=account_asset_management&version=14.0

Normally, all of these get automatically detected from github, populated in this Odoo, synchronized automatically for updates from Github. Synchronization is still in progress, now facing some delay due to rate limit.

Feel free to let me know your thoughts, thanks.

[OSevangelist]

Hi @jcdrubay i just had the time to review your initial proposal and the protoype that you've built. I really like the idea of encouraging people to contribute by making already existing PRs, forks and alike more readily visible.

On the other hand i don't believe, that making things more readily visible automatically increases the likelihood of final improvements, merges and alike, because it still needs effort and budget to finalize such "hacks" to become a mature contribution to OpenUpgrade.

As for the reasoning about it, i pretty much agree with what @rvalyi was saying about the chronic customer underestimation of overall migration effort (i just had a really funny case today). Nevertheless i hope we gonna have a next round of discussion with heavy contributors on OpenUpgrade (such as @legalsylvain @StefanRijnhart @pedrobaeza @MiquelRForgeFlow and others) and i believe we will be able to also briefly discuss your idea / prototype in that context. If your are interested in taking part in this call, please reach out to either of them or myself.

[thomaspaulb]

@jcdrubay Following the discussion on the contributors mailinglist I ended up here and read your arguments. As for my position I work with Openupgrade for our customers and have occasionally contributed a PR.

While your solution makes it a bit easier to find unmerged/unfinished Openupgrade code, I believe the real problem that we are facing pertains to all OCA repositories and not only this one - it's the combination of a somewhat high contribution barrier combined with the lack of budget/manpower available to match the speed of churnout of new Odoo releases. And this is not addressed by your solution.

I wonder if there is any solution though. What seemed to work well for some years is that OCA itself came together and (crowd)funded the development of OpenUpgrade for the next version, so that a dedicated team could get paid to take up the challenge of taking whatever's lying around (possibly helped by your tool) and make it mature and mergeable.