OpenJDK java/util/concurrent/Executors/UnreferencedExecutor OutOfMemory

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_sanity.openjdk_ppc64_aix_Nightly_testList_0/539 - p8-java1-ibm09
jdk_util_1
java/util/concurrent/Executors/UnreferencedExecutor.java

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_sanity.openjdk_ppc64_aix_Nightly_testList_0/539/openjdk_test_output.tar.gz

20:55:07  command: main -Xmx32m UnreferencedExecutor
20:55:07  reason: User specified action: run main/othervm -Xmx32m UnreferencedExecutor 
20:55:07  started: Fri Sep 19 00:30:37 UTC 2025
20:55:07  Mode: othervm [/othervm specified]
20:55:07  finished: Fri Sep 19 00:30:47 UTC 2025
20:55:07  elapsed time (seconds): 10.061
20:55:07  configuration:
20:55:07  STDOUT:
20:55:07  STDERR:
20:55:07  JVMDUMP055I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError", exception "Java heap space" at 2025/09/19 00:30:43 - please wait.

[pshipton]

@dmitripivkine fyi

[dmitripivkine]

System core hasn't been stored. Snap traces, javacore and GC verbose log are available.
Formal failure reason is detection of Excessive GC.

[pshipton]

Another one. Perhaps something has changed for these to start failing.

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_sanity.openjdk_ppc64_aix_Nightly_testList_0/540/ - p8-java1-ibm03

The tests run with -Xdump:system:none -Xdump:heap:none so no core will be created unless we change how it's run in a grinder.

[pshipton]

See if it's reproducible in a simple grinder, on build 611.
https://openj9-jenkins.osuosl.org/job/Grinder/4681/ - failed on p8-java1-ibm08

[pshipton]

It failed in the grinder, there is a core file captured. Set as a blocker since this doesn't seem intermittent.

Changes in the first build where it failed.
66256b9...1ee805a
eclipse-openj9/openj9-omr@0fe78e1...9922848
ibmruntimes/openj9-openjdk-jdk21@9796dc4...9fde630

[pshipton]

I see #22633 is a P specific change.
@rmnattas @zl-wang pls check

[pshipton]

Maybe not this P change because it was backported and I don't see the failure in the 0.55 nightly build. Unless it's interacting with something else that is in the head stream but not 0.55.
https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-JDK25-0.55/26/

[dmitripivkine]

Largest memory consuming types of the objects in the heap (based on core from grinder)):

          197772              9493056  java/util/concurrent/ConcurrentHashMap$Node
          196609             9437232  java/lang/ref/WeakReference
          25                      6321936  [Ljava/util/concurrent/ConcurrentHashMap$Node;

	 Total size of objects: 26824336
	 Total number of objects: 409375

[pshipton]

3x x 5 grinder internally (build 611), only failed once on paix823

[pshipton]

30x (6x x 5) open grinder on the original failing build (610) didn't fail. On 2,4,5,10,12 wrong options, failure is with -XX:-UseCompressedOops

30x open grinder on the build before the original failure didn't fail
The failure is intermittent, but fails very often on the open machines.
5 machine grinder failed iteration: 1, 5, 1, 2, 1

two 30x (6x x 5) internal grinders without #21043 didn't fail.
30x open grinder passed without #21043

[pshipton]

@fengxue-IS @keithc-ca @tajila fyi this is caused by #21043