Repository navigation
Closed
Description
Failure link
From an internal build job/Test_openjdk18_j9_extended.openjdk_ppc64_aix/3/(paix907):
openjdk version "18" 2022-03-22
IBM Semeru Runtime Open Edition 18.0.0.0-m1 (build 18+35)
Eclipse OpenJ9 VM 18.0.0.0-m1 (build openj9-0.31.0-m1, JRE 18 AIX ppc64-64-Bit Compressed References 20220210_28 (JIT enabled, AOT enabled)
OpenJ9 - 700cde7b8
OMR - bb5d44b8c
JCL - cdd4205c649 based on jdk-18+35)
Rerun in Grinder - Change TARGET to run only the failed test targets.
Optional info
Failure output (captured from console output)
[2022-02-11T07:42:13.549Z] variation: Mode150
[2022-02-11T07:42:13.549Z] JVM_OPTIONS: -XX:+UseCompressedOops
[2022-02-11T08:08:10.204Z] TEST: java/nio/channels/FileChannel/LargeGatheringWrite.java
[2022-02-11T08:08:10.205Z] STDOUT:
[2022-02-11T08:08:10.205Z] Seed from RandomFactory = -8247103834647789495L
[2022-02-11T08:08:10.205Z] Writing 5368709122 bytes of data...
[2022-02-11T08:08:10.205Z] STDERR:
[2022-02-11T08:08:10.205Z] java.lang.OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)
[2022-02-11T08:08:10.205Z] at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
[2022-02-11T08:08:10.205Z] at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
[2022-02-11T08:08:10.205Z] at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332)
[2022-02-11T08:08:10.205Z] at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:243)
[2022-02-11T08:08:10.205Z] at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:209)
[2022-02-11T08:08:10.205Z] at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:165)
[2022-02-11T08:08:10.205Z] at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:317)
[2022-02-11T08:08:10.205Z] at java.base/java.nio.channels.FileChannel.write(FileChannel.java:436)
[2022-02-11T08:08:10.205Z] at LargeGatheringWrite.main(LargeGatheringWrite.java:91)
[2022-02-11T08:08:10.205Z] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
[2022-02-11T08:08:10.205Z] at java.base/java.lang.reflect.Method.invoke(Method.java:577)
[2022-02-11T08:08:10.205Z] at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
[2022-02-11T08:08:10.205Z] at java.base/java.lang.Thread.run(Thread.java:889)
[2022-02-11T08:08:10.205Z]
[2022-02-11T08:08:10.205Z] JavaTest Message: Test threw exception: java.lang.OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)
[2022-02-11T08:08:10.206Z] TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)
job/Test_openjdk18_j9_extended.openjdk_s390x_linux/5/(ub18s390xrt-1-9)
[2022-02-11T07:08:46.817Z] variation: Mode650
[2022-02-11T07:08:46.817Z] JVM_OPTIONS: -XX:-UseCompressedOops
[2022-02-11T07:14:29.129Z] TEST: java/nio/channels/FileChannel/LargeGatheringWrite.java
[2022-02-11T07:14:29.131Z] TEST RESULT: Failed. Unexpected exit from test [exit code: 137]
[2022-02-11T07:14:29.131Z] --------------------------------------------------
[2022-02-11T07:34:06.023Z] Test results: passed: 445; failed: 1
[2022-02-11T07:34:23.488Z] Report written to /home/jenkins/workspace/Test_openjdk18_j9_extended.openjdk_s390x_linux/jvmtest/openjdk/report/html/report.html
[2022-02-11T07:34:23.488Z] Results written to /home/jenkins/workspace/Test_openjdk18_j9_extended.openjdk_s390x_linux/aqa-tests/TKG/output_16445604291340/jdk_nio_1/work
[2022-02-11T07:34:23.488Z] Error: Some tests failed or other problems occurred.
[2022-02-11T07:34:23.488Z]
[2022-02-11T07:34:23.488Z] jdk_nio_1_FAILED
job/Test_openjdk18_j9_extended.openjdk_x86-64_windows/5/(win16x64rt-1-2)
[2022-02-11T08:03:13.395Z] variation: Mode150
[2022-02-11T08:03:13.833Z] JVM_OPTIONS: -XX:+UseCompressedOops
openjdk version "18" 2022-03-22
IBM Semeru Runtime Open Edition 18.0.0.0-m1 (build 18+35)
Eclipse OpenJ9 VM 18.0.0.0-m1 (build openj9-0.31.0-m1, JRE 18 Windows Server 2016 amd64-64-Bit Compressed References 20220211_28 (JIT enabled, AOT enabled)
OpenJ9 - 700cde7b8
OMR - bb5d44b8c
JCL - cdd4205c649 based on jdk-18+35)
[2022-02-11T08:09:00.188Z] TEST: java/nio/channels/FileChannel/LargeGatheringWrite.java
[2022-02-11T08:09:00.190Z] STDERR:
[2022-02-11T08:09:00.190Z] java.lang.OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)
[2022-02-11T08:09:00.190Z] at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
[2022-02-11T08:09:00.190Z] at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
[2022-02-11T08:09:00.190Z] at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332)
[2022-02-11T08:09:00.190Z] at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:243)
[2022-02-11T08:09:00.190Z] at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:209)
[2022-02-11T08:09:00.190Z] at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:165)
[2022-02-11T08:09:00.190Z] at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:317)
[2022-02-11T08:09:00.190Z] at java.base/java.nio.channels.FileChannel.write(FileChannel.java:436)
[2022-02-11T08:09:00.190Z] at LargeGatheringWrite.main(LargeGatheringWrite.java:91)
[2022-02-11T08:09:00.190Z] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
[2022-02-11T08:09:00.190Z] at java.base/java.lang.reflect.Method.invoke(Method.java:577)
[2022-02-11T08:09:00.190Z] at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
[2022-02-11T08:09:00.190Z] at java.base/java.lang.Thread.run(Thread.java:889)
[2022-02-11T08:09:00.190Z]
[2022-02-11T08:09:00.190Z] JavaTest Message: Test threw exception: java.lang.OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)
Activity
[-]JDK18 AIX java/nio/channels/FileChannel/LargeGatheringWrite.java - OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)[/-][+]JDK18 java/nio/channels/FileChannel/LargeGatheringWrite.java - OutOfMemoryError: Cannot reserve 536870912 bytes of direct buffer memory (allocated: 3221225520, limit: 3758096384)[/+]tajila commentedon Feb 17, 2022
Looking at the test, it writes 3Gbs of data, half of the data comes from a heap buffer the other half from a direct buffer.
When attempting to write the contents of the heap buffer
java.base/sun.nio.ch.FileChannelImpl.writewill allocate a temporary direct buffer, this is where we fail with OOM, m guess is that we have reached the memory limit.tajila commentedon Feb 17, 2022
@JasonFengJ9 Does this fail on all platforms?
pshipton commentedon Feb 17, 2022
A couple days ago I started running a few open grinders on this test (just 1 iteration). It failed on AIX but not alinux or zlinux.
pshipton commentedon Feb 24, 2022
@tajila any progress on this one?
JasonFengJ9 commentedon Feb 24, 2022
Oops missed this, it also failed at Windows -
job/Test_openjdk18_j9_extended.openjdk_x86-64_windows/8/(win16x64rt-1-7)tajila commentedon Mar 1, 2022
The memory limit is determined by
j9gc_get_maximum_heap_size. The new requested amount is 48bytes over the the limittajila commentedon Mar 1, 2022
The test is run with Xmx4g, @dmitripivkine how is the max heap sized determined based on Xmx?
3 remaining items
pshipton commentedon Mar 1, 2022
Could OpenJDK have changed the default limit to be higher?
tajila commentedon Mar 1, 2022
On xlinux
with Xmx4g
on windows
The cause is that the
sun.nio.MaxDirectMemorySizeproperty is set to 3.7g on windows but it doesnt appear to be set on xlinuxtajila commentedon Mar 1, 2022
The tests above were done with openj9-jdk18
pshipton commentedon Mar 1, 2022
There is code to set it at 7/8 of the heap size https://github.com/eclipse-openj9/openj9/blob/master/runtime/jcl/common/system.c#L476
On xlinux 18 I see the system property is set (with -Xmx4g)
sun.nio.MaxDirectMemorySize = 3758096384pshipton commentedon Mar 1, 2022
With OpenJDK (8 - 18) I don't see any default setting for MaxDirectMemorySize
pshipton commentedon Mar 1, 2022
The OpenJ9 code comes from #2629
pshipton commentedon Mar 1, 2022
I don't see why OpenJ9 needs to set the limit at 7/8, seems it should just not set MaxDirectMemorySize at all.
pshipton commentedon Mar 1, 2022
This is weird because from the code it appears it's set the same on every platform, and I can see the property set on xlinux.
tajila commentedon Mar 1, 2022
May mistake, I was using the RI JDK18. On j9 xlinux it is set as well
pshipton commentedon Mar 1, 2022
We should validate that OpenJDK uses Runtime.getRuntime().totalMemory() as the default maxDirectMemory() for all versions, and if that is confirmed then remove setting sun.nio.MaxDirectMemorySize to be 7/8 of the max heap size, and just use the default. This doesn't need to occur for the Java 18 release, we can defer to 0.32.
pshipton commentedon Mar 3, 2022
I'll keep this open until we decide if we are going to backport the fix.
pshipton commentedon Mar 3, 2022
Fix is merged in head, 0.31, 0.32