Git submodules or Ansible Galaxy

[gonzaloetjo]

This decision would change how we use the collections on tdp-getting-started. Please add your arguments in the comments.

Git submodules:

Pros:

• Customization and development: easier to modify the source code of the collection if you need to make custom changes that aren't available in the official version and push them.
• Less work: Already used, and will continue to be used for tdp-vagrant, tdp-lib, tdp-ui and tdp-server.

Cons:

• Difficulty in management: Updating or changing submodules is not very straightforward.

Ansible Galaxy:

Pros:

• Simplicity: Provides a straightforward way to discover, share, and use Ansible collections through the requirements.yml file.
• Automated dependency management: Not used at the moment, but could be useful in the future.
• Declarative control:

collections:
  - name: https://devin-source.rte-france.com/bdp/tosit/tdp-collection.git
    type: git
    version: 0.0.1 # branch or tag or commit
  - name: https://devin-source.rte-france.com/bdp/tosit/tdp-collection-extras.git
    type: git
    version: 0.0.1 # branch or tag or commit
  - name: https://devin-source.rte-france.com/bdp/tosit/tdp-collection-prerequisites.git
    type: git
    version: 0.0.1 # branch or tag or commit

Cons:

• Development: More complicated to update changes in remote repositories when using installed roles or collections.
• Extra work: We have to update tdp-getting-started accordingly to utilize Ansible Galaxy.

[wdavidw]

Ansible is very specific, what if we need to register libraries such as a script to initialize users or datasets, what if you wish to implement custom commands or plugins?
Git submodules is commonly avoided, it never was embraced by the community like svn externals were.

I propose a package manager like poetry. We did it in the past with Ryba is never had to complain from day 1. Over time, it has proved to be simple, standardized, efficient, predictable, and comprehensible.

[rpignolet]

In the TDP ecosystem, we have a repository allowing deployment via Vagrant, Ansible collections, projects in python managed with Poetry, a Next.js project managed with NPM.

Each of these uses a different package manager.

To develop in TDP, it seems easier to me to have a parent repository that has all the necessary repositories in git submodules. This allows each developer to have a consistent state across all repositories when developing features that span multiple repositories. The disadvantages are the manual management of the dependencies of each repository and the mastery of git submodules.

In my opinion, we should make a new project like getting started which would be a repository for TDP developers in git submodule mode to be able to develop efficiently. tdp-dev?

Then in a new getting started, do it again by removing the git submodules to use the different package managers to install the different elements to deploy a TDP cluster.

[PaulFarault]

I'm not sure to understand what would contain this new getting-started. Could you develop a bit more please?

[rpignolet]

In the new getting started there might be a requirements.yaml file for Ansible dependencies, a requirements.txt for Python dependencies like tdp-lib and tdp-server, a package.json for the tdp-ui dependency.

A use of package managers to install what is needed.

[gonzaloetjo]

I'm having issues with the role of the new getting-started repository. We have three distinct types of users within TDP:

• Developers - contributors who want to make modifications.
• Explorers - individuals that want to see what TDP is all about, and want to check it first hand in their machines with an easy and straightforward installation.
• Users - organizations or groups who actively utilize TDP.

Is the intention for the getting-started repository to primarily cater to the 'Explorers' group? I'm concerned about the potential redundancy and added workload of maintaining two separate repositories for Developers and Explorers. There could be a significant overlap between these two categories, and there's nothing to ease and manage tdp for final users.

[rpignolet]

I also prefer to maintain only one repository but there is a problem with using git submodules. I don't have a problem with not using them anymore but I don't see any other way to have a central repository which allows to develop/contribute on all TDP repositories.

For me, the current getting started makes it possible to respond to developers and explorers. Developers can finely configure the different repositories in order to develop the functionalities. Explorers can deploy TDP without knowledge of git submodules thanks to the setup script.

On the other hand for the users, it is necessary to use package managers to install the various elements to deploy TDP (ansible-galaxy for the collections, pip for tdp-lib + tdp-server and npm for tdp-ui).

In any case, at the DGFiP we will industrialize and automate the installation of TDP Manager (tdp-lib + tdp-server + tdp-ui) and ansible collections. For the moment we have not started and the question will be whether we share this automation at the TOSIT level or if it is too specific to the needs of each user.

[wdavidw]

In Ryba/Masson/Nikita, we simply use the npm link command which creates symbolic links between NPM dependencies, it is particularly convenient inside a monorepo as everything is linked transparently. We never had to use git submodules and it proved to be simple and efficient. Not sure if Python and Poetry have the equivalent.