404 from https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin

[Strongbeard]

When I add the config from the README.adoc to my settings.gradle and build.gradle it tells me that it cannot find the plugin. I have additionally tried combinations of:

• Changing the plugin version to "2.2.0" from "2.3.0-SNAPSHOT"
• Changing the URL to https://dlmiles.github.io/gradle-xjc-plugin/java8/maven2/ and https://dlmiles.github.io/gradle-xjc-plugin/java11/maven2/

I noticed that when I try to curl https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin directly I get a 404 status code back with a body of "404 page not found".
I tested that the github token I generated works with curl -v -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/user/issues.

If github does not supply a consistent way to distribute the plugin, is it possible to publish it to the Gradle Plugin Repository?

[dlmiles]

Thanks for your report, sorry for the slow reply I have just noticed your issue while running through various repos checking up on them.

It is my intention at the next major update to publish this in the Gradle Plugin Repository, maybe also Maven Central.
One of the issues I need to clear up is exactly what GAV axis it will be published as externally and understand better maybe the Gradle plugin descriptor package and the gradle plugin implementation package split. At this time the worst case scenario looks like a timescale over the next 4 months. Now I know there is some 3rd party interest in doing this, I can attempt to speed up the timeline.

The infrastructure has slowly been put in place within this project towards the above goal, I was however providing sufficient time for the upstream author to collaborate on the upstream patch submissions to potentially respond favourably to get things merged upstream.

---

The distribution location that is reliable should be:
https://github.com/dlmiles?tab=packages&repo_name=gradle-xjc-plugin
But maybe my instruction are incorrect, missing details, or assume a view of the repo that has push access.

It is a pain as GH does not provide a browsable UI to help diagnostics.

However if we look at the packages published with this project from the link just above, then select one of the packages, then on the right hand side is a list of published files.

The information on this screen can be turned into a Maven2 style path such as below:

wget --header "Authorization: bearer $GITHUB_TOKEN" "https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin/org/darrylmiles/forked/org/unbroken-dome/gradle-xjc-plugin/2.2.0/gradle-xjc-plugin-2.2.0.pom"

wget --header "Authorization: bearer $GITHUB_TOKEN" "https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin/org/unbroken-dome/xjc/org.unbroken-dome.xjc.gradle.plugin/2.2.0/org.unbroken-dome.xjc.gradle.plugin-2.2.0.pom"

Note the different groupId names in use here, something I need to consider/fix/resolve before I publish to Gradle Portal.

This should then result in the download of the artifact. The GITHUB_TOKEN needs the scope read:packages enabled, maybe this can be done via the Settings -> Developer Settings -> Personal Access Token -> Tokens (classic) -> Generate new token (Generate new token (classic)) -> TICK [read:packages] -> Generate Token

I have validated this works with another GH account (that is not dlmiles).

This wget exercise is just to prove the packages are there at the usual GAV path when it is well formed, and you pass GH authentication requirements for access.

---

I have a project in my repo that consumes this project https://github.com/dlmiles/java-schema-ipxact maybe the configuration from this can be looked at to work out how to consume it. If my instructions are incorrect or missing some details or I have a custom $HOME/.gradle setting I did not document doing the heavy lifting.

TL;DR version:

git clone https://github.com/dlmiles/java-schema-ipxact
cd java-schema-ipxact
export GITHUB_USERNAME="your_username"
export GITHUB_TOKEN="your_token"  # must have read:packages scope
./gradlew --refresh-dependencies clean  # force all deps to be pulled in now
unset GITHUB_USERNAME # not needed now
unset GITHUB_TOKEN
./gradlew clean assemble check

I have not perform much external testing, taking a fresh 3rd party system (and a GH auth that is not my account) to validate all the instructions work correctly.

The closest thing that exists is another project in my GH repos list, that uses a workflow to build itself using the packages published by this project.

Here are some snippets that may help from a the gradle project that is externally hosted and built:

settings.gradle:

pluginManagement {
   // We are using this version of gradle-xjc-plugin because 2.0.0 does not
   // support all the building modes we use.  Pull requests are upstream.
   repositories {
       // 3rd party releases and SNAPSHOTs
       maven {
           //url "https://dlmiles.github.io/gradle-xjc-plugin/java8/maven2"
       url = "https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin"
           content {
               // this repository *only* contains artifacts for specific groups
               includeGroup "org.unbroken-dome.xjc"
               includeGroup "org.unbroken-dome"

               includeGroup "io.github.dlmiles.xjc"
               includeGroup "io.github.dlmiles"

               includeGroup "org.darrylmiles.forked.org.unbroken-dome.xjc"
               includeGroup "org.darrylmiles.forked.org.unbroken-dome"
           }
           credentials {
               // github requires any valid credentials even to GET packages
               username = System.getenv("GITHUB_USERNAME")
               password = System.getenv("GITHUB_TOKEN")
           }
       }

       // Official releases only
       gradlePluginPortal()
   }
}

ipxact-1685-2014/build.gradle

plugins {
    id 'java'
    id 'org.unbroken-dome.xjc' version '2.2.0'    // this maybe the misleading detail in README.adoc
    // the intention behind the last release is what has been documented, but what actually works today is this
}

I then have this github workflow action to manage the local locating of the gradle plugins (note the GITHUB_TOKEN will have permissions: package: write from workflow.yml config which implies package:read):
https://github.com/dlmiles/java-schema-ipxact/blob/master/.github/workflows/build.yml#L80

     # Resolve external dependencies (this will validate org.darrylmiles.forked.org.unbroken-dome.xjc is accessible)
      - name: Gradle - refresh-dependencies
        env:
          GITHUB_USERNAME: ${{ github.actor }}
          GITHUB_TOKEN:    ${{ secrets.GITHUB_TOKEN }}
        run: ./gradlew --refresh-dependencies clean

---

The other locations such as the maven2 part of gh-pages while it is there and exists, is unreliable for use directly from maven/gradle. It may only have the latest version up there at one time, it may disappear if GH disable project after 3 months of no activity, etc....

I started using the GH Maven package publish function when I started to make versioned releases as a method towards ironing out the details for publishing to Gradle Plugin Portal and/or Maven Central.

HTH

[dlmiles]

Feedback to self to address at next maintenance.

There is no public published version of 2.3.0-SNAPSHOT or anything *-SNAPSHOT so ideally the front page documentation should display the latest GA release version at all times, maybe a dev branch that is not the default branch is used to contain the patches adding *-SNAPSHOT, until they are merged to the default branch during release.

Resolve the Gradle Plugin descriptor groupId to be that which is currently published org.darrylmiles.forked.org.unbroken-dome.xjc but currently it only works on the original org.unbroken-dome.xjc because that is not something I had fully understood at the time of making the change.

I should then republish additional packages copies for back-versioned releases under the new Gradle Plugin descriptor GAV so there is a complete set under a single GAV, before looking at publishing it wider (GPP/MavenCentral). This is so that it should be possible for users to use the upstream 2.0.0 release and switch to this 2.0.0 and see no net change. Before then upgrading and migrating to newer releases.

[Strongbeard]

Sorry for taking so long to respond. I finally got the green light to revisit this after many months of delays.

I was able to get the plugin to work with the configuration you provided with 1 caveat:

I had to manually load the plugin's jar, module, and pom files into my maven local repo using mvn install:install-file -Dfile=... -DpomFile=... to get it to work. For some reason, gradle (v8.10.2) is timing-out while downloading the jar from maven.pkg.github.com, even though your wget example and any internet browser downloads it without issue. Maybe this timeout issue will resolve itself for me once this plugin is published to GPP or MavenCentral? In the meantime, I am planning on experimenting with caching it in a maven central mirror on my network.

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project '██████████'.
> Could not resolve all artifacts for configuration '██████████:classpath'.
   > Could not download gradle-xjc-plugin-2.2.0.jar (org.darrylmiles.forked.org.unbroken-dome:gradle-xjc-plugin:2.2.0)
      > Could not get resource 'https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin/org/darrylmiles/forked/org/unbroken-dome/gradle-xjc-plugin/2.2.0/gradle-xjc-plugin-2.2.0.jar'.
         > Read timed out

See this log excerpt for full stacktrace

2024-10-30T██:██:██.███-████ [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess] Error while accessing remote repository GitHub Packages gradle-xjc-plugin. Waiting 2000ms before next retry. 1 retries left
org.gradle.internal.resolve.ArtifactResolveException: Could not download gradle-xjc-plugin-2.2.0.jar (org.darrylmiles.forked.org.unbroken-dome:gradle-xjc-plugin:2.2.0)
	at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver$RemoteRepositoryAccess.resolveArtifact(ExternalResourceResolver.java:484)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository$ResolveAndCacheRepositoryAccess.resolveArtifact(CachingModuleComponentRepository.java:463)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.lambda$resolveArtifact$12(ErrorHandlingModuleComponentRepository.java:169)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.tryResolveAndMaybeDisable(ErrorHandlingModuleComponentRepository.java:236)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.performOperationWithRetries(ErrorHandlingModuleComponentRepository.java:193)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.ErrorHandlingModuleComponentRepository$ErrorHandlingModuleComponentRepositoryAccess.resolveArtifact(ErrorHandlingModuleComponentRepository.java:167)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.FilteredModuleComponentRepository$FilteringAccess.resolveArtifact(FilteredModuleComponentRepository.java:117)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainArtifactResolver.resolveArtifactLater(RepositoryChainArtifactResolver.java:75)
	at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.RepositoryChainArtifactResolver.lambda$resolveArtifact$0(RepositoryChainArtifactResolver.java:63)
	at org.gradle.internal.model.CalculatedValueContainerFactory$SupplierBackedCalculator.calculateValue(CalculatedValueContainerFactory.java:78)
	at org.gradle.internal.model.CalculatedValueContainer$CalculationState.lambda$attachValue$0(CalculatedValueContainer.java:228)
	at org.gradle.internal.Try.ofFailable(Try.java:50)
	at org.gradle.internal.model.CalculatedValueContainer$CalculationState.attachValue(CalculatedValueContainer.java:223)
	at org.gradle.internal.model.CalculatedValueContainer.finalizeIfNotAlready(CalculatedValueContainer.java:194)
	at org.gradle.internal.model.CalculatedValueContainer.finalizeIfNotAlready(CalculatedValueContainer.java:185)
	at org.gradle.api.internal.artifacts.ivyservice.resolveengine.artifact.ArtifactBackedResolvedVariant$DownloadArtifactFile.run(ArtifactBackedResolvedVariant.java:189)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:166)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationExecutor$QueueWorker.execute(DefaultBuildOperationExecutor.java:161)
	at org.gradle.internal.operations.DefaultBuildOperationQueue$WorkerRunnable.runOperation(DefaultBuildOperationQueue.java:272)
	at org.gradle.internal.operations.DefaultBuildOperationQueue$WorkerRunnable.doRunBatch(DefaultBuildOperationQueue.java:253)
	at org.gradle.internal.operations.DefaultBuildOperationQueue$WorkerRunnable.lambda$runBatch$0(DefaultBuildOperationQueue.java:238)
	at org.gradle.internal.resources.AbstractResourceLockRegistry.whileDisallowingLockChanges(AbstractResourceLockRegistry.java:50)
	at org.gradle.internal.work.DefaultWorkerLeaseService.whileDisallowingProjectLockChanges(DefaultWorkerLeaseService.java:235)
	at org.gradle.internal.operations.DefaultBuildOperationQueue$WorkerRunnable.lambda$runBatch$1(DefaultBuildOperationQueue.java:238)
	at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:263)
	at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:127)
	at org.gradle.internal.operations.DefaultBuildOperationQueue$WorkerRunnable.runBatch(DefaultBuildOperationQueue.java:224)
	at org.gradle.internal.operations.DefaultBuildOperationQueue$WorkerRunnable.run(DefaultBuildOperationQueue.java:192)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:48)
	at java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base@17.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base@17.0.11/java.lang.Thread.run(Thread.java:840)
Caused by: org.gradle.api.resources.ResourceException: Could not get resource 'https://maven.pkg.github.com/dlmiles/gradle-xjc-plugin/org/darrylmiles/forked/org/unbroken-dome/gradle-xjc-plugin/2.2.0/gradle-xjc-plugin-2.2.0.jar'.
	at org.gradle.internal.resource.ResourceExceptions.failure(ResourceExceptions.java:74)
	at org.gradle.internal.resource.ResourceExceptions.getFailed(ResourceExceptions.java:57)
	at org.gradle.internal.resource.transfer.AbstractExternalResourceAccessor.withContent(AbstractExternalResourceAccessor.java:41)
	at org.gradle.internal.resource.transfer.DefaultExternalResourceConnector.withContent(DefaultExternalResourceConnector.java:59)
	at org.gradle.internal.resource.transfer.ProgressLoggingExternalResourceAccessor$DownloadOperation.call(ProgressLoggingExternalResourceAccessor.java:123)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:209)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
	at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:166)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
	at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
	at org.gradle.internal.resource.transfer.ProgressLoggingExternalResourceAccessor.withContent(ProgressLoggingExternalResourceAccessor.java:46)
	at org.gradle.internal.resource.transfer.AccessorBackedExternalResource.withContentIfPresent(AccessorBackedExternalResource.java:103)
	at org.gradle.internal.resource.transfer.DefaultCacheAwareExternalResourceAccessor.copyToCache(DefaultCacheAwareExternalResourceAccessor.java:188)
	at org.gradle.internal.resource.transfer.DefaultCacheAwareExternalResourceAccessor.lambda$getResource$1(DefaultCacheAwareExternalResourceAccessor.java:86)
	at org.gradle.cache.internal.ProducerGuard$AdaptiveProducerGuard.guardByKey(ProducerGuard.java:97)
	at org.gradle.internal.resource.transfer.DefaultCacheAwareExternalResourceAccessor.getResource(DefaultCacheAwareExternalResourceAccessor.java:80)
	at org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver.downloadByUrl(DefaultExternalResourceArtifactResolver.java:109)
	at org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver.downloadStaticResource(DefaultExternalResourceArtifactResolver.java:92)
	at org.gradle.api.internal.artifacts.repositories.resolver.DefaultExternalResourceArtifactResolver.resolveArtifact(DefaultExternalResourceArtifactResolver.java:62)
	at org.gradle.api.internal.artifacts.repositories.resolver.ExternalResourceResolver$RemoteRepositoryAccess.resolveArtifact(ExternalResourceResolver.java:477)
	... 35 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:288)
	at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:314)
	at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:355)
	at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:808)
	at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:484)
	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478)
	at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
	at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1465)
	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1069)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:197)
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
	at org.gradle.internal.resource.transfer.AbstractProgressLoggingHandler$ProgressLoggingInputStream.read(AbstractProgressLoggingHandler.java:63)
	at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:282)
	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
	at com.google.common.io.CountingInputStream.read(CountingInputStream.java:64)
	at java.base/java.io.FilterInputStream.read(FilterInputStream.java:106)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1487)
	at org.apache.commons.io.IOUtils.copy(IOUtils.java:1107)
	at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1460)
	at org.gradle.internal.resource.transfer.DownloadAction.execute(DownloadAction.java:59)
	at org.gradle.internal.resource.transfer.AccessorBackedExternalResource.lambda$withContentIfPresent$2(AccessorBackedExternalResource.java:105)
	at org.gradle.internal.resource.transfer.ProgressLoggingExternalResourceAccessor$DownloadOperation.lambda$call$0(ProgressLoggingExternalResourceAccessor.java:130)
	at org.gradle.internal.resource.transfer.AbstractExternalResourceAccessor.withContent(AbstractExternalResourceAccessor.java:39)
	... 54 more

[dlmiles]

@Strongbeard Have you configure Maven authentication ? This can be added on a server by server basis. From memory the <id> need to match the repository ID.

https://maven.apache.org/settings.html#servers in $HOME/.m2/settings.xml although maybe an issue to keep updated if the GITHUB_TOKEN expires, maybe a readonly (with only the scope read:packages) PAT token with a 1 year expiry would work better.

GH PAT link https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens