Repository navigation
Open
Labels
comp:jitproject:MHUsed to track Method Handles related workUsed to track Method Handles related worksegfaultIssues that describe segfaults / JVM crashesIssues that describe segfaults / JVM crashestest failure
Milestone
Description
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_xl_Nightly/217
jdk/internal/reflect/CallerSensitive/MissingCallerSensitive.java
ub18-ppcle-1
00:13:49 stderr:
00:13:49 Unhandled exception
00:13:49 Type=Segmentation error vmState=0x00000000
00:13:49 J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
00:13:49 Handler1=00007AEB926F0690 Handler2=00007AEB9262C100
00:13:49 R0=00007AEB9275F548 R1=00007AEB4D93B660 R2=00007AEB92896F00 R3=0000000000000004
00:13:49 R4=00007AEB6C1B05B0 R5=00007AEB8C496A00 R6=00007AEB1D70E250 R7=0000000000000498
00:13:49 R8=0000000000000018 R9=0000000000000018 R10=0000000000000008 R11=00007AEB6C2C8800
00:13:49 R12=0000000000004400 R13=00007AEB4D9468E0 R14=00007AEAA800D080 R15=00007AEAB0006F00
00:13:49 R16=00007AEB4E170038 R17=FFFFFFFFFFFFFFFF R18=00007AEB4D93BDA8 R19=00007AEB1D70E5D6
00:13:49 R20=00007AEAA8000C50 R21=0000000000000010 R22=00007AEAA800D028 R23=00007AEB93824410
00:13:49 R24=00007AEAB0006F00 R25=00007AEB4D93F240 R26=00007AEB4D93B9A0 R27=00007AEB4D93B890
00:13:49 R28=00007AEB4D93B870 R29=00007AEB6C2B88B0 R30=0000000000000000 R31=0000000000000000
00:13:49 NIP=00007AEB927583E8 MSR=800000010280F033 ORIG_GPR3=00000000000081C8 CTR=00007AEB927635E0
00:13:49 LINK=00007AEB9275F548 XER=0000000000000000 CCR=0000000044004482 SOFTE=0000000000000001
00:13:49 TRAP=0000000000000300 DAR=0000000000000020 dsisr=0000000040000000 RESULT=0000000000000000
00:13:49 FPR0 00007aeaa800d0c8 (f: 2818625792.000000, d: 6.677211e-310)
00:13:49 FPR1 40551ab4a0000000 (f: 2684354560.000000, d: 8.441727e+01)
00:13:49 FPR2 3fe8000000000000 (f: 0.000000, d: 7.500000e-01)
00:13:49 FPR3 004100540045004d (f: 4522061.000000, d: 1.891455e-307)
00:13:49 FPR4 3fce840b4ac4e4d2 (f: 1254417664.000000, d: 2.384047e-01)
00:13:49 FPR5 bfe7154748bef6c8 (f: 1220474624.000000, d: -7.213475e-01)
00:13:49 FPR6 3fe62e42fefa39ef (f: 4277811712.000000, d: 6.931472e-01)
00:13:49 FPR7 3fc526e57720db08 (f: 1998641920.000000, d: 1.652495e-01)
00:13:49 FPR8 bfdffffef20a4123 (f: 4060758272.000000, d: -4.999997e-01)
00:13:49 FPR9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR10 bfe03f2a28285f5e (f: 673734464.000000, d: -5.077105e-01)
00:13:49 FPR11 3ff0000000000000 (f: 0.000000, d: 1.000000e+00)
00:13:49 FPR12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
00:13:49 FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
00:13:49 Module=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_xl_Nightly/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
00:13:49 Module_base_address=00007AEB926B0000
00:13:49 Target=2_90_20201130_358 (Linux 4.15.0-96-generic)
00:13:49 CPU=ppc64le (16 logical CPUs) (0x1fdc70000 RAM)
00:13:49 ----------- Stack Backtrace -----------
00:13:49 (0x00007AEB927583E8 [libj9vm29.so+0xa83e8])
00:13:49 (0x00007AEB927B6134 [libj9vm29.so+0x106134])
00:13:49 (0x00007AEB926C8284 [libj9vm29.so+0x18284])
00:13:49 (0x00007AEB92756110 [libj9vm29.so+0xa6110])
00:13:49 (0x00007AEB9262D508 [libj9prt29.so+0x3d508])
00:13:49 (0x00007AEB927517C4 [libj9vm29.so+0xa17c4])
00:13:49 (0x00007AEB925C1268 [libj9thr29.so+0x11268])
00:13:49 (0x00007AEB937E885C [libpthread.so.0+0x885c])
00:13:49 clone+0x98 (0x00007AEB93699028 [libc.so.6+0x159028])
Metadata
Metadata
Assignees
Labels
comp:jitproject:MHUsed to track Method Handles related workUsed to track Method Handles related worksegfaultIssues that describe segfaults / JVM crashesIssues that describe segfaults / JVM crashestest failure
Activity
pshipton commentedon Dec 1, 2020
@tajila @gacholio fyi
gacholio commentedon Dec 1, 2020
I'll take a look.
gacholio commentedon Dec 1, 2020
2 cores, both crash in the exact same place dereferencing NULL somewhere in the bytecode interpreter. Neither java stack is immediately useful.
gacholio commentedon Dec 1, 2020
Crash appears to be in:
at bytecode 6.
gacholio commentedon Dec 1, 2020
Full stack:
gacholio commentedon Dec 1, 2020
The WAITSTATUS static appears valid:
gacholio commentedon Dec 1, 2020
The stack is modified by the VarHandle implementation code:
The VarHandle is pushed onto the stack, and it's old stack slot is overwritten with a MethodHandle (somehow 0 in this case, which is no doubt the cause of the crash).
gacholio commentedon Dec 1, 2020
I expect the actual crash occurs transitioning to the MH interpreter due to the NULL MH. The code which set the slot to NULL is:
https://github.com/eclipse/openj9/blob/master/runtime/vm/BytecodeInterpreter.hpp#L8469-L8553
gacholio commentedon Dec 1, 2020
@babsingh There must be a timing hole somewhere in the caching mechanism, though I don't see it. This might also be related to weak memory ordering on PPC. I notice the callSiteType variable has been randomly marked volatile, which might trick the compiler into doing something stupid.
pshipton commentedon Dec 1, 2020
100x grinder https://ci.eclipse.org/openj9/job/Grinder/1231/
gacholio commentedon Dec 1, 2020
Grinder appears to be failing for an unrelated reason:
pshipton commentedon Dec 1, 2020
Ya, I messed up the test name.
100x (25x on 4 machines) grinder https://ci.eclipse.org/openj9/job/Grinder/1232/ - passed
27 remaining items
lzhou2025 commentedon Jul 7, 2026
From other Ubuntu/ppc64le machine, using the jpackcore files, I can get a gdb bt. To make gdb show source, gdb also needs the right debug info files (.debuginfo) from that build.
lzhou2025 commentedon Jul 7, 2026
For example, from my local master build, the debug info files:
lzhou2025 commentedon Jul 7, 2026
The target machine is
Ubuntu 24.04 LTS - 4, I think I need access to that machine, because internal machines seem all Ubuntu 22.pshipton commentedon Jul 7, 2026
I think it's fine, you just aren't doing it correctly.
I extracted the stack trace on a rhel8 ppcle64 machine, using the jpackcore result.
core.20260704.033908.550927.0001.dmp.zipin my current directoryhome/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_Nightly_testList_2/jdkbinary/j2sdk-imagefrom https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Build_JDK11_ppc64le_linux_Nightly/1274/OpenJ9-JDK11-ppc64le_linux-20260704-012122.tar.gzgdb -c core.20260704.033908.550927.0001.dmp home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_Nightly_testList_2/jdkbinary/j2sdk-image/bin/javapshipton commentedon Jul 7, 2026
You can populate
home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_Nightly_testList_2/jdkbinary/j2sdk-imagein your current directory, you don't need to recreate that location from root.pshipton commentedon Jul 7, 2026
You should try it yourself, but here is the stack (done on rhel8le-rtp-rt8-1).
lzhou2025 commentedon Jul 8, 2026
I downloaded the debuginfo files, will try later once the machine is available. From your gdb bt, the thread caused the crash:
pshipton commentedon Jul 8, 2026
You don't need any particular machine, if you do what I described any ppcle64 machine will do.
lzhou2025 commentedon Jul 8, 2026
I got bt by following above steps, Thanks!
But gdb complaints about "No such file or directory", where to get the right version of source codes ?
pshipton commentedon Jul 8, 2026
github, using the shas from the -version output
babsingh commentedon Jul 8, 2026
Just from the top frame of the native stack, this crash appears specific to the OpenJ9 MethodHandle implementation and occurs during an interpreter-to-JIT (i2j) transition. I’m therefore adding the
comp:jitlabel and removing thecomp:vmlabel.The failure is highly intermittent, and we are already working toward enabling the OJDK MethodHandle implementation. Given that direction, it does not make sense to dedicate resources to fixing the OpenJ9 MethodHandle implementation at this point. Enabling the OJDK MethodHandle implementation in OpenJ9 JDK 11 should implicitly resolve this issue, since that path does not depend on the OpenJ9 MethodHandle code involved in this crash.
@lzhou2025 As a learning exercise, you can open the core file in
jdmpviewand further study the stack slots for the failing thread. That should provide more insight into the failure symptoms and help with resolving similar issues in the future.openj9/runtime/oti/VMHelpers.hpp
Lines 376 to 382 in cb98417
@0xdaryl @nbhuiyan @tajila, FYI.
lzhou2025 commentedon Jul 8, 2026
Here is java stack dump from jdmpview,
gacholio commentedon Jul 9, 2026
At a guess, this is likely caused by a bad object as the MethodHandle.