Plugin documentation doesn't get updates

[curveenk]

Service(s)

plugins.jenkins.io

Summary

We observed that our plugin page is not getting updated for 2 latest release(s) starting from last week:
https://plugins.jenkins.io/eggplant-runner/

• The documentation changes does not reflect
• Plugin version is not updated

The two versions (0.0.1.191.v72dea_07931b_6 and 0.0.1.189.v1e3397db_cee8) are already available for install for users but the page isn't in sync with the latest updates.
https://github.com/jenkinsci/eggplant-runner-plugin/releases/tag/0.0.1.191.v72dea_07931b_6
https://github.com/jenkinsci/eggplant-runner-plugin/releases/tag/0.0.1.189.v1e3397db_cee8

Could you help to advise what could be wrong and where else should we check for the status for page update?

Thanks in advance!

@jenkins-infra/hosting

[dduportal]

@timja @halkeye I need your help on this one. I don't know where to look (how is the plugin site generated? Is it the docker image or is there a bucket with the static content?)

[timja]

unsure, would need reverse engineering, it used to be on trusted ci but maybe have moved to infra.ci

[NotMyFault]

No idea if it's related, but the plugin-site job on infra.ci is failing since yesterday.

[dduportal]

No idea if it's related, but the plugin-site job on infra.ci is failing since yesterday.

Yep, last succeessful build is the 12th of october. I assume it might be related?

(the build fails when resolving the NPM dependencies it seems)

[curveenk]

Can we have a rough estimate of time when it could be up?

[halkeye]

When someone has a minute to look at the failing job. I can't access the vpn while at $dayjob, so I won't be able to look at it for at least 8 hours.

[halkeye]

Was able to find time after jenkins board meeting ended early.

So looks like bad data.

https://plugins.jenkins.io/api/plugin/alibabacloud-ecs is returning "null" for wiki url
seems to be {} on https://updates.jenkins.io/current/plugin-documentation-urls.json as well.

I didn't even know that was possible. Fix could be applied to update center (probably a good long term solution). Plugin site api (boo, want to get rid of this) or plugin site.

https://github.com/jenkinsci/alibabacloud-ecs-plugin/blob/master/pom.xml#L215-L222
https://github.com/jenkinsci/alibabacloud-credentials-plugin/blob/master/pom.xml doesn't have a <url> at all.

Pinging @zbynek incase he has a minute to fix this before I would.

[halkeye]

We should at least make a plugin-site issue though.

[zbynek]

I'll check

[zbynek]

@halkeye do we know what caused it? The wiki URL is null also for https://github.com/jenkinsci/r7insight-log-forwarder-plugin/ which was not released in 4 years, so this is not the source data but some change in infra (update center change? NodeJS update on infra? ...)

[halkeye]

Last success was Build #1075 (Oct 12, 2022, 3:11:00 PM)

The only update I see to update-center in october was jenkins-infra/update-center2@33273a6

plugin site is still using "jenkinsciinfra/builder:2.0.55@sha256:77dd8139b078d307e3dcb7f68dbf6d57b357776229ccaafb93cbb5a54a07d349" for builds and we explicitly pin the version in jenkinsfile, which hasn't been updated in a while.

Nothing i can see is really different.

Looking at the error again, those plugins were not likely the error, its one of the ones not finished, the first failed job alibabacloud succeeded.

[halkeye]

So adding debugging into the code (will probably PR it soon)

Error: Undefined url for alibabacloud-credentials
which was released 8 days ago, which seems to be before this started failing...
https://github.com/jenkinsci/alibabacloud-credentials-plugin/releases/tag/alibabacloud-credentials-1.2

I wasn't running it properly locally

[halkeye]

nope, still same error.

curl -qs https://updates.jenkins.io/current/plugin-documentation-urls.json | jq 'to_entries[] | select(.value | has("url") | not)'
{
  "key": "alibabacloud-credentials",
  "value": {}
}
{
  "key": "alibabacloud-ecs",
  "value": {}
}
{
  "key": "codebeamer-xunit-uploader",
  "value": {}
}
{
  "key": "lambdatest-automation",
  "value": {}
}
{
  "key": "r7insight-log-forwarder",
  "value": {}
}
{
  "key": "vrealize-automation-8",
  "value": {}
}

I'm not sure when or how it changed, but those 6 are breaking the plugin site generator. Shoudn't be too hard to flat out ignore them.

[zbynek]

so should we merge jenkins-infra/plugin-site#1244 ?

[halkeye]

for record keeping.

Build #1075 (Oct 12, 2022, 3:11:00 PM)

was the last successful build.

@daniel-beck do you know anything upstream with the data that might have changed that would have caused it to have empty objects in the documentation urls json?

[zbynek]

investigation so far: wiki URL falls back to SCM url, but for these 6 extensions even SCM url is missing from update-center-actual.json. Since SCM URL can't be obtained from their POMs, update-center2 should use this fallback:
https://github.com/jenkins-infra/update-center2/blob/33273a62564d5259b9dfdfd6f891fe812d32eee4/src/main/java/io/jenkins/update_center/HPI.java#L478 but that somehow fails (?) GitHub API problem?

[halkeye]

Its possible all the upgrades to RPU's dependacies broke it. is https://ci.jenkins.io/job/Infra/job/repository-permissions-updater/job/master/lastSuccessfulBuild/artifact/permissions/plugin-alibabacloud-ecs.yml/*view*/ being used at all?

[daniel-beck]

do you know anything upstream with the data that might have changed that would have caused it to have empty objects in the documentation urls json?

I switched update site generation to the branch for jenkins-infra/update-center2#646 on Sep 28 (and once again forgot about it 😞). Switched it back to master just now, but this wouldn't align with your timeline. Otherwise update-center2 has been very low activity recently.

[halkeye]

The only thing that lines up is all the depenancy updates to Rpu but I can't figure out how that would be involve.

Site generation is fixed but we don't know the root cause.

[daniel-beck]

@zbynek might be on to something though, the content of plugin-documentation-urls.json did change since September 28 (had a build "kept forever" from then).

[curveenk]

Thanks for all the help and quick actions :) We are now able to see the changes reflected on the plugin page. Can anyone help me to understand what is being fixed and if we can assume it will never fall back to previous state again?

[zbynek]

@curveenk the metadata of plugins is processed by Update Center and this data is used to build the Plugin Site. For some plugins that do not provide all the metadata Update Center started returning "null" recently instead of giving reasonable fallback. Plugin Site then failed to build, so information was missing for all plugins, not just those with incomplete metadata. Now the Plugin Site build is more robust, so this should no longer happen (and I think this ticket can be closed). What still needs to be figured out is why Update Center no longer uses the fallback values.

[curveenk]

@zbynek Great! Thanks for clarifying it. I can close this ticket now, as the issue has been resolved.

Again, thank you.

[daniel-beck]

@zbynek

I think this ticket can be closed

Are we tracking the investigation somewhere?