HADOOP-15760. Upgrade commons-collections to commons-collections4 #7097

NihalJain · 2024-10-03T09:17:08Z

This moves Hadoop to Apache commons-collections4.
Apache commons-collections has been removed and is completely banned from the source code.

Contributed by Nihal Jain

Description of PR

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

NihalJain · 2024-10-03T09:19:11Z

Backports #7006
Changes are almost same, except that we had to do extra exclusions for commons-collection from hbase 1.7.1 and hadoop 2.8.4. None of this would have been needed if we had on branch-3.4 #7017

CC: @steveloughran

NihalJain

Added details on new changes added as part of backport

hadoop-project/pom.xml

...roject/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/pom.xml

steveloughran

+1 pending a successful Yetus build.

steveloughran · 2024-10-03T10:39:53Z

so this PR fixes up the imports so that hbase 1.x exlusions are coped with.

what if we cut hbase 1 from branch-3.4 first?

NihalJain · 2024-10-03T10:48:01Z

so this PR fixes up the imports so that hbase 1.x exlusions are coped with.

There were no direct imports due to hbase 1.x, hence no code change was needed, hence I have only excluded common-collections from the affected hbase 1.x / hadoop2.x jars . Lets see how tests go through with this change.

what if we cut hbase 1 from branch-3.4 first?

In that case we could simply cherry-pick changes from trunk with no extra changes needed for handling common-collections transitively coming from hbase 1.x / hadoop 2.x

hadoop-yetus · 2024-10-04T06:16:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 9 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	4m 10s		Maven dependency ordering for branch
+1 💚	mvninstall	42m 22s		branch-3.4 passed
+1 💚	compile	18m 29s		branch-3.4 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	18m 13s		branch-3.4 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	5m 4s		branch-3.4 passed
+1 💚	mvnsite	21m 58s		branch-3.4 passed
+1 💚	javadoc	9m 47s		branch-3.4 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	8m 14s		branch-3.4 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗	spotbugs	0m 18s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	34m 50s	/branch-spotbugs-root-warnings.html	root in branch-3.4 has 4 extant spotbugs warnings.
+0 🆗	spotbugs	0m 26s		branch/hadoop-client-modules/hadoop-client-minicluster no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	2m 47s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in branch-3.4 has 1 extant spotbugs warnings.
-1 ❌	spotbugs	1m 25s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html	hadoop-hdfs-project/hadoop-hdfs-rbf in branch-3.4 has 1 extant spotbugs warnings.
+1 💚	shadedclient	39m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	60m 24s		the patch passed
+1 💚	compile	19m 54s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javac	19m 54s		the patch passed
+1 💚	compile	16m 50s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	javac	16m 50s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 19s		the patch passed
+1 💚	mvnsite	17m 43s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	9m 15s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	8m 26s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗	spotbugs	0m 20s		hadoop-project has no data from spotbugs
+0 🆗	spotbugs	0m 22s		hadoop-client-modules/hadoop-client-minicluster has no data from spotbugs
+1 💚	shadedclient	75m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	758m 43s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 44s		The patch does not generate ASF License warnings.
		1257m 55s

Reason	Tests
Failed junit tests	hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageEntities
	hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageDomain
	hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRunCompaction
	hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageApps
	hadoop.yarn.server.timelineservice.storage.TestHBaseTimelineStorageSchema
	hadoop.yarn.server.timelineservice.storage.TestTimelineReaderHBaseDown
	hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowRun
	hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
	hadoop.yarn.server.timelineservice.storage.TestTimelineWriterHBaseDown
	hadoop.yarn.server.timelineservice.storage.flow.TestHBaseStorageFlowActivity
	hadoop.hdfs.TestFileChecksum

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/1/artifact/out/Dockerfile
GITHUB PR	#7097
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux d2d056b6f15a 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / `889a25e`
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/1/testReport/
Max. process+thread count	3556 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp hadoop-tools/hadoop-aws hadoop-tools/hadoop-aliyun hadoop-tools/hadoop-sls hadoop-client-modules/hadoop-client-minicluster . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

NihalJain · 2024-10-04T06:27:08Z

Failures are related. Getting following:

java.lang.NoClassDefFoundError: org/apache/commons/collections/map/UnmodifiableMap
	at org.apache.hadoop.conf.Configuration$DeprecationContext.<init>(Configuration.java:457)
	at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:497)
	at org.apache.hadoop.hbase.io.compress.Compression$Algorithm.<init>(Compression.java:302)
	at org.apache.hadoop.hbase.io.compress.Compression$Algorithm.<init>(Compression.java:105)
	at org.apache.hadoop.hbase.io.compress.Compression$Algorithm$1.<init>(Compression.java:106)
	at org.apache.hadoop.hbase.io.compress.Compression$Algorithm.<clinit>(Compression.java:106)
	at org.apache.hadoop.hbase.HBaseTestingUtility.<clinit>(HBaseTestingUtility.java:213)
	at org.apache.hadoop.yarn.server.timelineservice.reader.AbstractTimelineReaderHBaseTestBase.setup(AbstractTimelineReaderHBaseTestBase.java:60)
	at org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage.setupBeforeClass(TestTimelineReaderWebServicesHBaseStorage.java:91)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.collections.map.UnmodifiableMap
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 29 more

Hadoop 2.8.4 uses common-collections:3.2.2 during runtime, so we cannot exclude it from hbase 1.7.x / hadoop 2.8.x dependecies.

I think we have two options here:

Remove all usages from current code path and still keep common-collections:3.2.2 for hbase 1.7.x / hadoop 2.8.x (Cherry picking trunk PR does that job. But we wont be able to get rid of common-collections:3.2.2 from our classpath
Wait for HADOOP-19107. Drop support for HBase v1 timeline service & upgrade HBase v2 (#6629). #7017 and then merge it, given we plan to merge that change into branch-3.4

steveloughran · 2024-10-09T12:57:05Z

merged #7017, so this patch gets easier

This moves Hadoop to Apache commons-collections4. Apache commons-collections has been removed and is completely banned from the source code. Contributed by Nihal Jain

NihalJain · 2024-10-12T11:05:34Z

Rebased code post #7017, dropping changes not needed any more. Now changes are exactly same as #7006

hadoop-yetus · 2024-10-13T06:49:37Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 9 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	4m 1s		Maven dependency ordering for branch
+1 💚	mvninstall	41m 57s		branch-3.4 passed
+1 💚	compile	17m 31s		branch-3.4 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	16m 30s		branch-3.4 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	4m 41s		branch-3.4 passed
+1 💚	mvnsite	23m 15s		branch-3.4 passed
+1 💚	javadoc	8m 55s		branch-3.4 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	7m 50s		branch-3.4 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗	spotbugs	0m 19s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	35m 24s	/branch-spotbugs-root-warnings.html	root in branch-3.4 has 3 extant spotbugs warnings.
+0 🆗	spotbugs	0m 25s		branch/hadoop-client-modules/hadoop-client-minicluster no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	1m 52s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html	hadoop-hdfs-project/hadoop-hdfs-rbf in branch-3.4 has 1 extant spotbugs warnings.
+1 💚	shadedclient	38m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	60m 2s		the patch passed
+1 💚	compile	20m 1s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javac	20m 1s		the patch passed
+1 💚	compile	19m 1s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	javac	19m 1s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	5m 0s		the patch passed
+1 💚	mvnsite	21m 53s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 42s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	8m 2s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗	spotbugs	0m 20s		hadoop-project has no data from spotbugs
+0 🆗	spotbugs	0m 25s		hadoop-client-modules/hadoop-client-minicluster has no data from spotbugs
+1 💚	shadedclient	64m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	732m 20s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 44s		The patch does not generate ASF License warnings.
		1220m 10s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
	hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/2/artifact/out/Dockerfile
GITHUB PR	#7097
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux 76f4cb5cfbc1 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / `46fb328`
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/2/testReport/
Max. process+thread count	3819 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp hadoop-tools/hadoop-aws hadoop-tools/hadoop-aliyun hadoop-tools/hadoop-sls hadoop-client-modules/hadoop-client-minicluster . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-10-14T02:40:03Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 9 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	4m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	39m 50s		branch-3.4 passed
+1 💚	compile	17m 39s		branch-3.4 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	compile	16m 7s		branch-3.4 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	checkstyle	4m 37s		branch-3.4 passed
+1 💚	mvnsite	24m 58s		branch-3.4 passed
+1 💚	javadoc	8m 52s		branch-3.4 passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	8m 1s		branch-3.4 passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗	spotbugs	0m 22s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	31m 27s	/branch-spotbugs-root-warnings.html	root in branch-3.4 has 3 extant spotbugs warnings.
+0 🆗	spotbugs	0m 30s		branch/hadoop-client-modules/hadoop-client-minicluster no spotbugs output file (spotbugsXml.xml)
-1 ❌	spotbugs	1m 22s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html	hadoop-hdfs-project/hadoop-hdfs-rbf in branch-3.4 has 1 extant spotbugs warnings.
+1 💚	shadedclient	33m 44s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	mvninstall	51m 34s		the patch passed
+1 💚	compile	17m 3s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javac	17m 3s		the patch passed
+1 💚	compile	16m 13s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+1 💚	javac	16m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 22s		the patch passed
+1 💚	mvnsite	19m 6s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	8m 36s		the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04
+1 💚	javadoc	7m 58s		the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05
+0 🆗	spotbugs	0m 21s		hadoop-project has no data from spotbugs
+0 🆗	spotbugs	0m 27s		hadoop-client-modules/hadoop-client-minicluster has no data from spotbugs
+1 💚	shadedclient	64m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	731m 16s		root in the patch passed.
+1 💚	asflicense	1m 45s		The patch does not generate ASF License warnings.
		1177m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/3/artifact/out/Dockerfile
GITHUB PR	#7097
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux c63503916f71 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / `46fb328`
Default Java	Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/3/testReport/
Max. process+thread count	3853 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-mapreduce-project/hadoop-mapreduce-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp hadoop-tools/hadoop-aws hadoop-tools/hadoop-aliyun hadoop-tools/hadoop-sls hadoop-client-modules/hadoop-client-minicluster . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7097/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

steveloughran

+1
it's a direct cherrypick now; lower risk

slfan1989 · 2024-10-17T04:53:37Z

LGTM +1.

NihalJain commented Oct 3, 2024

View reviewed changes

steveloughran reviewed Oct 3, 2024

View reviewed changes

HADOOP-15760. Upgrade commons-collections to commons-collections4

46fb328

This moves Hadoop to Apache commons-collections4. Apache commons-collections has been removed and is completely banned from the source code. Contributed by Nihal Jain

NihalJain force-pushed the HADOOP-15760_branch-3.4 branch from 889a25e to 46fb328 Compare October 12, 2024 10:28

steveloughran approved these changes Oct 15, 2024

View reviewed changes

slfan1989 approved these changes Oct 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-15760. Upgrade commons-collections to commons-collections4 #7097

HADOOP-15760. Upgrade commons-collections to commons-collections4 #7097

NihalJain commented Oct 3, 2024 •

edited

Loading

NihalJain commented Oct 3, 2024

NihalJain left a comment

steveloughran left a comment

steveloughran commented Oct 3, 2024

NihalJain commented Oct 3, 2024 •

edited

Loading

hadoop-yetus commented Oct 4, 2024

NihalJain commented Oct 4, 2024

steveloughran commented Oct 9, 2024

NihalJain commented Oct 12, 2024

hadoop-yetus commented Oct 13, 2024

hadoop-yetus commented Oct 14, 2024

steveloughran left a comment

slfan1989 commented Oct 17, 2024

HADOOP-15760. Upgrade commons-collections to commons-collections4 #7097

Are you sure you want to change the base?

HADOOP-15760. Upgrade commons-collections to commons-collections4 #7097

Conversation

NihalJain commented Oct 3, 2024 • edited Loading

Description of PR

How was this patch tested?

For code changes:

NihalJain commented Oct 3, 2024

NihalJain left a comment

Choose a reason for hiding this comment

steveloughran left a comment

Choose a reason for hiding this comment

steveloughran commented Oct 3, 2024

NihalJain commented Oct 3, 2024 • edited Loading

hadoop-yetus commented Oct 4, 2024

NihalJain commented Oct 4, 2024

steveloughran commented Oct 9, 2024

NihalJain commented Oct 12, 2024

hadoop-yetus commented Oct 13, 2024

hadoop-yetus commented Oct 14, 2024

steveloughran left a comment

Choose a reason for hiding this comment

slfan1989 commented Oct 17, 2024

NihalJain commented Oct 3, 2024 •

edited

Loading

NihalJain commented Oct 3, 2024 •

edited

Loading