Work around problems with flattened pom #95

daniel-beck · 2024-10-12T10:14:20Z

Not a long-term fix, at minimum we would need to keep updating to the latest weekly. A little code reorg can probably easily take care of the problem by passing in the previously downloaded core, but right now I'm looking for confirmation that the dependency:copy approach works.

Attempts to address #58 (not using a magic term due to uncertainty).

Since repo.jenkins-ci.org no longer proxies all of central, I needed to change the included settings.xml to make this work for me.

Seems to have worked reasonably well with a fairly small JSON locally, now waiting for PR build result.

cat update-center.actual.json

{
	"connectionCheckUrl":"https://www.google.com/",
	"core":{
		"buildDate":"Oct 08, 2024",
		"name":"core",
		"sha1":"LbsOmdwPbpQLFtrfP/hjzZ7BFlY=",
		"sha256":"ucsqDWSJgF2nsLGYze3wBPAlHrCEi2z4ISh1Xna12T0=",
		"size":96029975,
		"url":"https://updates.jenkins.io/download/war/2.480/jenkins.war",
		"version":"2.480"
	},
	"deprecations":{},
	"generationTimestamp":"2024-10-12T10:02:08Z",
	"id":"default",
	"plugins":{
		"bouncycastle-api":{
			"buildDate":"Jul 05, 2024",
			"dependencies":[],
			"developers":[
				{
					"developerId":"alobato",
					"name":"Alvaro Lobato"
				},
				{
					"developerId":"dnusbaum",
					"name":"Devin Nusbaum"
				},
				{
					"developerId":"teilo",
					"name":"James Nord"
				},
				{
					"developerId":"danielbeck",
					"name":"Daniel Beck"
				},
				{
					"developerId":"kevingrdj",
					"name":"Kevin Guerroudj"
				},
				{
					"developerId":"wfollonier",
					"name":"Wadeck Follonier"
				},
				{
					"developerId":"alecharp",
					"name":"Adrien Lecharpentier"
				},
				{
					"developerId":"amuniz",
					"name":"Antonio Muñiz"
				},
				{
					"developerId":"aneveux",
					"name":"Antoine Neveux"
				},
				{
					"developerId":"batmat",
					"name":"Baptiste Mathus"
				},
				{
					"developerId":"bzzitsme",
					"name":"Temirlan Dyussyumbayev"
				},
				{
					"developerId":"carroll",
					"name":"Carroll Chiou"
				},
				{
					"developerId":"jpochat",
					"name":"Jérôme Pochat"
				},
				{
					"developerId":"kerogers",
					"name":"Kenneth Rogers"
				},
				{
					"developerId":"mikecirioli",
					"name":"mike cirioli"
				},
				{
					"developerId":"olamy",
					"name":"Olivier Lamy"
				},
				{
					"developerId":"pldi23",
					"name":"Dmitry Platonov"
				},
				{
					"developerId":"rarabaolaza",
					"name":"Raul Arabaolaza"
				},
				{
					"developerId":"rsandell",
					"name":"rsandell"
				},
				{
					"developerId":"twasyl",
					"name":"Thierry Wasylczenko"
				},
				{
					"developerId":"vlatombe",
					"name":"Vincent Latombe"
				}
			],
			"excerpt":"This plugin provides a stable API to Bouncy Castle related tasks.",
			"gav":"org.jenkins-ci.plugins:bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_",
			"issueTrackers":[
				{
					"reportUrl":"https://www.jenkins.io/participate/report-issue/redirect/#21482",
					"type":"jira",
					"viewUrl":"https://issues.jenkins.io/issues/?jql=component=21482"
				}
			],
			"labels":[
				"library"
			],
			"name":"bouncycastle-api",
			"popularity":276418,
			"previousTimestamp":"2024-07-05T16:00:10.00Z",
			"previousVersion":"2.30.1.78.1-246.ve1089fe22055",
			"releaseTimestamp":"2024-07-05T17:28:50.00Z",
			"requiredCore":"2.361.4",
			"scm":"https://github.com/jenkinsci/bouncycastle-api-plugin",
			"sha1":"6ze2/L+TglJMeIlC255yFH5Oz1I=",
			"sha256":"BSxDdShFiAapRk1ZpXcULcXIhI4waxnLqGXgJ9GtIDE=",
			"size":8853632,
			"title":"bouncycastle API",
			"url":"https://updates.jenkins.io/download/plugins/bouncycastle-api/2.30.1.78.1-248.ve27176eb_46cb_/bouncycastle-api.hpi",
			"version":"2.30.1.78.1-248.ve27176eb_46cb_",
			"wiki":"https://plugins.jenkins.io/bouncycastle-api"
		},
		"credentials":{
			"buildDate":"Oct 11, 2024",
			"compatibleSinceVersion":"1354",
			"dependencies":[
				{
					"name":"configuration-as-code",
					"optional":true,
					"version":"1850.va_a_8c31d3158b_"
				},
				{
					"name":"bouncycastle-api",
					"optional":false,
					"version":"2.30.1.78.1-248.ve27176eb_46cb_"
				},
				{
					"name":"structs",
					"optional":false,
					"version":"338.v848422169819"
				}
			],
			"developers":[
				{
					"developerId":"jglick",
					"name":"Jesse Glick"
				},
				{
					"developerId":"kohsuke",
					"name":"Kohsuke Kawaguchi"
				},
				{
					"developerId":"oleg_nenashev",
					"name":"Oleg Nenashev"
				},
				{
					"developerId":"jvz",
					"name":"Matt Sicker"
				},
				{
					"developerId":"danielbeck",
					"name":"Daniel Beck"
				},
				{
					"developerId":"kevingrdj",
					"name":"Kevin Guerroudj"
				},
				{
					"developerId":"wfollonier",
					"name":"Wadeck Follonier"
				},
				{
					"developerId":"alecharp",
					"name":"Adrien Lecharpentier"
				},
				{
					"developerId":"amuniz",
					"name":"Antonio Muñiz"
				},
				{
					"developerId":"aneveux",
					"name":"Antoine Neveux"
				},
				{
					"developerId":"batmat",
					"name":"Baptiste Mathus"
				},
				{
					"developerId":"bzzitsme",
					"name":"Temirlan Dyussyumbayev"
				},
				{
					"developerId":"carroll",
					"name":"Carroll Chiou"
				},
				{
					"developerId":"jpochat",
					"name":"Jérôme Pochat"
				},
				{
					"developerId":"kerogers",
					"name":"Kenneth Rogers"
				},
				{
					"developerId":"mikecirioli",
					"name":"mike cirioli"
				},
				{
					"developerId":"olamy",
					"name":"Olivier Lamy"
				},
				{
					"developerId":"pldi23",
					"name":"Dmitry Platonov"
				},
				{
					"developerId":"rarabaolaza",
					"name":"Raul Arabaolaza"
				},
				{
					"developerId":"rsandell",
					"name":"rsandell"
				},
				{
					"developerId":"teilo",
					"name":"James Nord"
				},
				{
					"developerId":"twasyl",
					"name":"Thierry Wasylczenko"
				},
				{
					"developerId":"vlatombe",
					"name":"Vincent Latombe"
				}
			],
			"excerpt":"This plugin allows you to store credentials in Jenkins.",
			"gav":"org.jenkins-ci.plugins:credentials:1389.vd7a_b_f5fa_50a_2",
			"issueTrackers":[
				{
					"reportUrl":"https://www.jenkins.io/participate/report-issue/redirect/#16523",
					"type":"jira",
					"viewUrl":"https://issues.jenkins.io/issues/?jql=component=16523"
				}
			],
			"labels":[],
			"name":"credentials",
			"popularity":275637,
			"previousTimestamp":"2024-10-07T00:15:08.00Z",
			"previousVersion":"1384.vf0a_2ed06f9c6",
			"releaseTimestamp":"2024-10-11T09:32:37.00Z",
			"requiredCore":"2.479",
			"scm":"https://github.com/jenkinsci/credentials-plugin",
			"sha1":"opZ7/0Wsun1frw07O0+FsCF9ExU=",
			"sha256":"RhoZ03WuSWwxLEQnSKgvXR4eKfwXFzkJusPtouNFO9I=",
			"size":632795,
			"title":"Credentials",
			"url":"https://updates.jenkins.io/download/plugins/credentials/1389.vd7a_b_f5fa_50a_2/credentials.hpi",
			"version":"1389.vd7a_b_f5fa_50a_2",
			"wiki":"https://plugins.jenkins.io/credentials"
		},
		"plain-credentials":{
			"buildDate":"Jun 19, 2024",
			"dependencies":[
				{
					"name":"credentials",
					"optional":false,
					"version":"1344.v5a_3f65a_1e173"
				}
			],
			"developers":[
				{
					"developerId":"jglick",
					"name":"Jesse Glick"
				},
				{
					"developerId":"danielbeck",
					"name":"Daniel Beck"
				},
				{
					"developerId":"kevingrdj",
					"name":"Kevin Guerroudj"
				},
				{
					"developerId":"wfollonier",
					"name":"Wadeck Follonier"
				},
				{
					"developerId":"alecharp",
					"name":"Adrien Lecharpentier"
				},
				{
					"developerId":"amuniz",
					"name":"Antonio Muñiz"
				},
				{
					"developerId":"aneveux",
					"name":"Antoine Neveux"
				},
				{
					"developerId":"batmat",
					"name":"Baptiste Mathus"
				},
				{
					"developerId":"bzzitsme",
					"name":"Temirlan Dyussyumbayev"
				},
				{
					"developerId":"carroll",
					"name":"Carroll Chiou"
				},
				{
					"developerId":"jpochat",
					"name":"Jérôme Pochat"
				},
				{
					"developerId":"kerogers",
					"name":"Kenneth Rogers"
				},
				{
					"developerId":"mikecirioli",
					"name":"mike cirioli"
				},
				{
					"developerId":"olamy",
					"name":"Olivier Lamy"
				},
				{
					"developerId":"pldi23",
					"name":"Dmitry Platonov"
				},
				{
					"developerId":"rarabaolaza",
					"name":"Raul Arabaolaza"
				},
				{
					"developerId":"rsandell",
					"name":"rsandell"
				},
				{
					"developerId":"teilo",
					"name":"James Nord"
				},
				{
					"developerId":"twasyl",
					"name":"Thierry Wasylczenko"
				},
				{
					"developerId":"vlatombe",
					"name":"Vincent Latombe"
				}
			],
			"excerpt":"Allows use of plain strings and files as credentials.",
			"gav":"org.jenkins-ci.plugins:plain-credentials:183.va_de8f1dd5a_2b_",
			"issueTrackers":[
				{
					"reportUrl":"https://www.jenkins.io/participate/report-issue/redirect/#18123",
					"type":"jira",
					"viewUrl":"https://issues.jenkins.io/issues/?jql=component=18123"
				}
			],
			"labels":[],
			"name":"plain-credentials",
			"popularity":271825,
			"previousTimestamp":"2024-05-14T16:18:08.00Z",
			"previousVersion":"182.v468b_97b_9dcb_8",
			"releaseTimestamp":"2024-06-19T11:22:56.00Z",
			"requiredCore":"2.426.3",
			"scm":"https://github.com/jenkinsci/plain-credentials-plugin",
			"sha1":"zIZQ4FkR3g8sXnq/92wR3rbD9MU=",
			"sha256":"lCLqp2XmWR48hFv+ycEF9WAKBYlRopQK7Fvg7XbqgTo=",
			"size":17837,
			"title":"Plain Credentials",
			"url":"https://updates.jenkins.io/download/plugins/plain-credentials/183.va_de8f1dd5a_2b_/plain-credentials.hpi",
			"version":"183.va_de8f1dd5a_2b_",
			"wiki":"https://plugins.jenkins.io/plain-credentials"
		},
		"structs":{
			"buildDate":"Jun 18, 2024",
			"dependencies":[],
			"developers":[
				{
					"developerId":"jglick",
					"name":"Jesse Glick"
				},
				{
					"developerId":"kohsuke",
					"name":"Kohsuke Kawaguchi"
				},
				{
					"developerId":"oleg_nenashev",
					"name":"Oleg Nenashev"
				},
				{
					"developerId":"abayer",
					"name":"Andrew Bayer"
				},
				{
					"developerId":"dnusbaum",
					"name":"Devin Nusbaum"
				},
				{
					"developerId":"rsandell",
					"name":"rsandell"
				},
				{
					"developerId":"carroll",
					"name":"Carroll Chiou"
				},
				{
					"developerId":"bitwiseman",
					"name":"Liam Newman"
				},
				{
					"developerId":"kshultz",
					"name":"Karl Shultz"
				}
			],
			"excerpt":"Library plugin for DSL plugins that need names for Jenkins objects.",
			"gav":"org.jenkins-ci.plugins:structs:338.v848422169819",
			"issueTrackers":[
				{
					"reportUrl":"https://www.jenkins.io/participate/report-issue/redirect/#21442",
					"type":"jira",
					"viewUrl":"https://issues.jenkins.io/issues/?jql=component=21442"
				}
			],
			"labels":[],
			"name":"structs",
			"popularity":277616,
			"previousTimestamp":"2024-01-18T03:15:16.00Z",
			"previousVersion":"337.v1b_04ea_4df7c8",
			"releaseTimestamp":"2024-06-18T13:54:12.00Z",
			"requiredCore":"2.414.3",
			"scm":"https://github.com/jenkinsci/structs-plugin",
			"sha1":"6K9qjPCCLfjiTPLOO+4b0O9jIRI=",
			"sha256":"fK6BGjl4j1ipVHdGMfCiedS/XjJnKDcmioU4Imeor2Y=",
			"size":43570,
			"title":"Structs",
			"url":"https://updates.jenkins.io/download/plugins/structs/338.v848422169819/structs.hpi",
			"version":"338.v848422169819",
			"wiki":"https://plugins.jenkins.io/structs"
		}
	},
	"updateCenterVersion":"1",
	"warnings":[]
}

MarkEWaite · 2024-10-12T10:41:44Z

Thanks so much for this! I think that the Jenkinsfile needs to be switched to run maven with Java 17. I've attempted that in build 3 on ci.jenkins.io using the "replay" feature of Pipeline.

Unfortunately, that didn't fix it either.

Now trying build 4 with the artifact caching proxy definition removed.

MarkEWaite · 2024-10-12T13:08:43Z

Build 4 completed successfully. The zip file that is generated includes many more entries than are displayed on the current index page. I've started a build 5 that uses the artifact caching proxy without the authentication content. Build 4 took 2 hours to complete. Hopefully build 5 will be faster and less burdensome on the artifact repositories.

MarkEWaite · 2024-10-12T13:13:26Z

we would need to keep updating to the latest weekly.

I know someone (@gounthar) that is very skilled with updatecli manifests. I can likely persuade him to write the mianifest that will keep that up to date.

daniel-beck · 2024-10-12T13:19:03Z

Not an unqualified success: A lot of projects seem to have HTTP repository URLs in their poms, which recent(ish) Maven releases block.

gitee depends on jbcrypt which isn't in releases, need to switch that to public.

daniel-beck · 2024-10-12T13:22:52Z

write the mianifest that will keep that up to date.

As I wrote, probably better done with code reorg once we know this approach isn't doomed to fail.

Wild guess so far: This will not work for any plugin with implied dependencies it actually uses (but might only result in fewer extensions discovered, unsure).

…or repo definitions

gounthar · 2024-10-12T13:27:07Z

we would need to keep updating to the latest weekly.

I know someone (@gounthar) that is very skilled with updatecli manifests. I can likely persuade him to write the mianifest that will keep that up to date.

😊
So kind of you, Mark. 🤗
I'll give it a try.

timja · 2024-10-12T15:29:55Z

Sorry accidentally pushed it your branch meant to be my fork. I'll finish testing but something like this should work.

timja · 2024-10-12T15:56:54Z

src/main/java/org/jenkinsci/extension_indexer/SourceAndLibs.java

-                "dependency:copy-dependencies",
+                "org.apache.maven.plugins:maven-dependency-plugin:3.8.0:copy-dependencies",
+                "org.apache.maven.plugins:maven-dependency-plugin:3.8.0:copy",
+                "-Dartifact=org.jenkins-ci.main:jenkins-core:2.479",


Tested in timja-org#3

could revert back to 2.480 although this will demonstrate it working.

Noting #95 (comment) for visibility just in case.

done in 67785f7

timja · 2024-10-12T16:07:29Z

Build 4 completed successfully. The zip file that is generated includes many more entries than are displayed on the current index page. I've started a build 5 that uses the artifact caching proxy without the authentication content. Build 4 took 2 hours to complete. Hopefully build 5 will be faster and less burdensome on the artifact repositories.

created #96

daniel-beck · 2024-10-13T07:30:09Z

This now looks a lot better than before.

Some content is missing or seemingly wrong compared to master (super old, its content pre-dates 2.421, as AppearanceCategory is missing):

macstadium-orka has an extension point (good) but no implementations found anymore (bad). OK: Looks like improper cleanup in version 2 of the plugin.
matlab is missing its page. OK: It no longer declared the extension point TestRunTypeList since 2.12.0.
windows-slaves is missing its page. OK: Plugin has been suspended.
Weird package change in dtkit plugin of implementations of an extension point declared in dtkit-api. This one looks weird, as there's no dependency (dtkit-api is a successor of dtkit).

So with one obsolete plugin (dtkit really should have been marked deprecated when it was archived), this looks reasonable, at least compared to the previous output.

While I'd like to confirm the guess in #95 (comment) this seems mergeable anyway, any further improvements can be done as followup after the extensions index is restored.

daniel-beck · 2024-10-13T11:46:34Z

super old, its content pre-dates 2.421

Filed as jenkins-infra/helpdesk#4342

Work around problems with flattened pom

fdee475

This was referenced Oct 12, 2024

https://www.jenkins.io/doc/developer/extensions/ missing plugins with ExtensionPoints #58

Open

Credential artifact-caching-proxy-credentials unavailable on ci.jenkins.io jenkins-infra/helpdesk#4340

Closed

Force version of dependency plugin that works, attempt to override po…

3139ca3

…or repo definitions

Add renovate

4de488a

Add registry url

ba418d4

timja reviewed Oct 12, 2024

View reviewed changes

timja mentioned this pull request Oct 12, 2024

Fix build #96

Merged

timja and others added 2 commits October 12, 2024 17:49

Merge branch 'master' into flattened-pom-workaround

4f875f6

Code re-org to pass down core version

67785f7

timja merged commit 52b7790 into jenkins-infra:master Oct 13, 2024
3 checks passed

daniel-beck mentioned this pull request Oct 13, 2024

Nobody noticed that the last successful extension-indexer run was in August 2023 jenkins-infra/helpdesk#4342

Open

daniel-beck deleted the flattened-pom-workaround branch October 13, 2024 11:46

This was referenced Oct 13, 2024

extension-indexer runs with wrong Java (11 instead of 17) on infra.ci jenkins-infra/helpdesk#4343

Closed

Extensions index is missing many implementations jenkins-infra/helpdesk#3746

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Work around problems with flattened pom #95

Work around problems with flattened pom #95

daniel-beck commented Oct 12, 2024 •

edited

Loading

MarkEWaite commented Oct 12, 2024 •

edited

Loading

MarkEWaite commented Oct 12, 2024 •

edited

Loading

MarkEWaite commented Oct 12, 2024

daniel-beck commented Oct 12, 2024

daniel-beck commented Oct 12, 2024

gounthar commented Oct 12, 2024

timja commented Oct 12, 2024

timja Oct 12, 2024

daniel-beck Oct 12, 2024

timja Oct 12, 2024 •

edited

Loading

timja commented Oct 12, 2024

daniel-beck commented Oct 13, 2024 •

edited

Loading

daniel-beck commented Oct 13, 2024

Work around problems with flattened pom #95

Work around problems with flattened pom #95

Conversation

daniel-beck commented Oct 12, 2024 • edited Loading

MarkEWaite commented Oct 12, 2024 • edited Loading

MarkEWaite commented Oct 12, 2024 • edited Loading

MarkEWaite commented Oct 12, 2024

daniel-beck commented Oct 12, 2024

daniel-beck commented Oct 12, 2024

gounthar commented Oct 12, 2024

timja commented Oct 12, 2024

timja Oct 12, 2024

Choose a reason for hiding this comment

daniel-beck Oct 12, 2024

Choose a reason for hiding this comment

timja Oct 12, 2024 • edited Loading

Choose a reason for hiding this comment

timja commented Oct 12, 2024

daniel-beck commented Oct 13, 2024 • edited Loading

daniel-beck commented Oct 13, 2024

daniel-beck commented Oct 12, 2024 •

edited

Loading

MarkEWaite commented Oct 12, 2024 •

edited

Loading

MarkEWaite commented Oct 12, 2024 •

edited

Loading

timja Oct 12, 2024 •

edited

Loading

daniel-beck commented Oct 13, 2024 •

edited

Loading