Shared Gradle Convention Plugins for Hiero

[jjohannes]

Problem

Many repositories in the github.com/hashgraph organization use Gradle to define their project structure and build (compile, test, check, ...) the code. The currently most sophisticated setup is the one in this repository hedera-services. In the past, the configuration in this repository was often copied when new repositories were created. This approach has a number of downsides:

• Copying, and then later not fixing, undiscovered issue
  In the past, copying was done at arbitrary points. Possibly copying "undiscovered issue" that were then only fixed here, but not in the copies. There is no mechanism in place to track or even know about the copies.
• Inconsistent structures
  Although the Gradle setup in hedera-services is "more stable" now, it is still in flux and there are areas to improve it further. In the past, larger changes were made from time to time (e.g. in which folder Gradle config is located). We now have inconsistency in the different repos, where the same things are named differently or are located in different locations.
• Maintainability
  Even if we would somehow improve tracking the "copies" it would be quite some work to manually update all places if we make a change
• Ease in setting up new repositories
  Right now, setting up a new repo requires someone with a lot of expertise with the Gradle setup and the new setup is already doomed to be "imperfect" from the beginning (e.g. containing, in that context, unused configuration code).

Solution

General solution idea

Gradle plugins can be published to Maven repositories, just as we publish Modules to repositories. Technically, any Maven repository can be used, but there is a "Maven Central for Gradle Plugins" called the "Gradle Plugin Portal" - https://plugins.gradle.org/ - that may be used to publish open-source plugins.

We use the approach of Convention Plugins to configure the Gradle setup already. Technically, Gradle makes no difference between Convention Plugins and other "types" of plugins. Hence, we could publish the plugins we have defined in gradle/plugins/src/main/kotlin in as they are. Then we can reuse them in other repositories.

Most of the convention plugins are not specific to hedera-services already. Which means they are almost good to be reused in another context/repository.

Concrete Proposal:

1. Go through the plugins in gradle/plugins/src/main/kotlin and make sure to have a set of plugins not specific for hedera-services. This should include splitting the plugins up further to have one plugin for each "aspect" of the build. Consistently name these plugins com.hedera.gradle.convention.<aspect>.gradle.kts. These plugins should work in a way that they can be used independently and individually. (While doing this, also improve documentation Document the Gradle build configuration #11568.)
   • com.hedera.gradle.conventions.java-compile
   • com.hedera.gradle.conventions.javadoc
   • com.hedera.gradle.conventions.test
   • com.hedera.gradle.conventions.shadow
   • com.hedera.gradle.conventions.jpms
   • com.hedera.gradle.conventions.jmh
   • com.hedera.gradle.conventions.spotless
   • ...
2. Copy move these plugins to a separate git repository - hedera-gradle-conventions
3. Locally use the plugins in the following repositories to test the approach. This can locally be done using the includeBuild("...") feature before actually publishing the plugins.
   • https://github.com/hashgraph/hedera-services (this repo)
   • https://github.com/hashgraph/hedera-sdk-java
   • https://github.com/hashgraph/hedera-block-node
   • https://github.com/hashgraph/hedera-cryptography
   • https://github.com/hashgraph/pbj
4. Publish the plugins, to the Gradle Plugin Portal. As the plugins are all opens source, it makes sense to publish them there. A plugin ID like com.hedera.gradle.convention.java-compile can be read as "Hedera's Gradle Conventions for Compiling Java Code" - which other projects (outside of Hedera) could also use. Similar to how we use code formatting styles defined by other orgs/companies. This should be done via GH actions in the new hedera-gradle-conventions repository.
5. Use the initial version of the published plugins in...
   • https://github.com/hashgraph/hedera-services
   • https://github.com/hashgraph/hedera-sdk-java
   • https://github.com/hashgraph/hedera-block-node
   • https://github.com/hashgraph/hedera-cryptography
   • https://github.com/hashgraph/pbj
     When this is done, other existing and new repositories can use the convention plugins. If further development on the plugins is done, a new version of the plugins is published. Repositories then individually can decide to updated (or not yet update) to new versions. Then it is clear for each repository which variation (i.e. which version) of our Gradle configuration they use, which is much clearer than what we have now.

Alternatives

No response

[nathanklick]

We should not publish from the hedera-services repository. These should be defined in their own repository and all publishing should happen via CI under our Gradle Plugins Repository SA.

[mxtartaglia-sl]

Nothing to add here; it is a great idea.
Also, I agree with Nathan that it should have its own repository.

[rbarkerSL]

I like this proposal.

[jjohannes]

Thank you all for the feedback. Doing this in a separate Git repo from the start as proposed by @nathanklick sounds good. Not only for the publishing (credentials) and pipeline, but also for having a dedicated place for the Gradle configuration and it's documentation. I think it will make doing documentation and discovering documentation simpler.

I'll adjust the proposal accordingly.

How should the repo be named?
https://github.com/hashgraph/hedera-services/hedera-gradle-conventions ?
With the publishing coordinates (for the Jar containing the plugins) being com.hedera:hedera-gradle-conventions?

[rbair23]

@jjohannes give me a few days to work up a good name for this repo.

[jjohannes]

I am making progress on this and would be ready to get started with the repo setup next week.

Any thoughts on the name @rbair23?

[jjohannes]

PRs to move to the common configuration:

• build: use common Hiero Gradle config #15282
• build: use common Hiero Gradle config hedera-block-node#216
• https://github.com/hashgraph/hedera-cryptography/pull/78
• build: use common Hiero Gradle config hedera-sdk-java#2033
• build: use common Hiero Gradle config pbj#302

[jjohannes]

As discussed between @rbarkerSL and @nathanklick this should go directly under the Hiero namespace.

Suggestion:

• Repository: https://github.com/LFDT-hiero/hiero-gradle-conventions
• Maven Coordinates: org.hiero:hiero-gradle-conventions
• Main plugin ID: org.hiero.gradle.build
• Specific plugins: org.hierero.gradle.[plugin-category].[plugin-name]
  • categories as described here.
  • Initially we have these.

When using the shared configuration in a repository, it is sufficient to use the main plugin in settings.gradle.kts with a version. All other plugins can then be used in build.gradle.kts files without version.

// settings.gradle.kts
plugins { id("org.hierero.gradle.build") version "0.1.0" }

...

[rbarkerSL]

Need to create a PR on hiero-ledger to add a new repository

[jjohannes]

https://github.com/hiero-ledger/hiero-gradle-conventions

[jjohannes]

This is done. Three repositories have been switched to the setup (hedera-services, hedera-cryptography, hedera-sdk-java). Moving more repos to the setup is tracked in separate PRs/issues.

Further improvements and fixes to the convention plugins are tracked here:
https://github.com/hiero-ledger/hiero-gradle-conventions/issues