Skip to content

JDK17+ : Try BootLoader if bootstrapClassLoader didn't find the native - #12981

Merged
pshipton merged 1 commit into
eclipse-openj9:masterfrom
JasonFengJ9:osxsecurity
Sep 13, 2021
Merged

pshipton merged 1 commit into
eclipse-openj9:masterfrom
JasonFengJ9:osxsecurity

Conversation

@JasonFengJ9

@JasonFengJ9 JasonFengJ9 commented Jun 16, 2021 •

Copy link
Copy Markdown
Member

Try BootLoader.getNativeLibraries().find(entryName) if bootstrapClassLoader.nativelibs.find(entryName) didn't succeed;
Add trace points;
Refactoring methods affected.

This fixes an internal test failure with following stack trace

java.lang.UnsatisfiedLinkError: apple/security/KeychainStore._scanKeychain()V
	at java.base/apple.security.KeychainStore.engineLoad(KeychainStore.java:769)
	at java.base/java.security.KeyStoreSpi.engineLoad(KeyStoreSpi.java:439)
	at java.base/java.security.KeyStoreSpi.engineLoad(KeyStoreSpi.java:403)
	at java.base/java.security.KeyStore.load(KeyStore.java:1506)
	at java.base/java.security.KeyStore$Builder$2$1.run(KeyStore.java:2193)
	at java.base/java.security.KeyStore$Builder$2$1.run(KeyStore.java:2182)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:774)
	at java.base/java.security.KeyStore$Builder$2.getKeyStore(KeyStore.java:2228)

Signed-off-by: Jason Feng fengj@ca.ibm.com

Comment thread runtime/jcl/common/vm_scar.c Resolved
@pshipton

Copy link
Copy Markdown
Member

@JasonFengJ9

Copy link
Copy Markdown
Member Author

Why isn't this working?

This is because the loading of libosxsecurity.dylib is requested after javaVM->applicationClassLoader initialization hence not registered as bootstrap library

openj9/runtime/j9vm/jvm.c

Lines 3677 to 3690 in 4462d50

if (NULL == javaVM->applicationClassLoader) {
J9NativeLibrary *nativeLibrary = NULL;
J9InternalVMFunctions *vmFuncs = javaVM->internalVMFunctions;
J9VMThread *currentThread = vmFuncs->currentVMThread(javaVM);
Assert_SC_notNull(currentThread);
vmFuncs->internalEnterVMFromJNI(currentThread);
vmFuncs->internalReleaseVMAccess(currentThread);
if (vmFuncs->registerBootstrapLibrary(currentThread, libName, &nativeLibrary, FALSE) == J9NATIVELIB_LOAD_OK) {
result = (void*)nativeLibrary->handle;
}
vmFuncs->internalAcquireVMAccess(currentThread);
vmFuncs->internalExitVMToJNI(currentThread);
Trc_SC_LoadLibrary_BootStrap(libName);
} else {

@pshipton

Copy link
Copy Markdown
Member

This seems like a hack, not a proper fix. The JCL code requests the BootLoader to load the library.

@JasonFengJ9

Copy link
Copy Markdown
Member Author

The native library loading approach has been changed for JDK16+. OpenJ9 VM only has the knowledge of libName

openj9/runtime/j9vm/jvm.c

Lines 3633 to 3653 in 620b0ca

/**
* void * JNICALL JVM_LoadLibrary(char *libName)
*
* Attempts to load the shared library specified by libName. If
* successful, returns the file handle, otherwise returns NULL.
*
* @param libName a null terminated string containing the libName.
* For Windows platform, this incoming libName is encoded as J9STR_CODE_WINDEFAULTACP,
* and is required to be converted to J9STR_CODE_MUTF8 for internal usages.
*
* @return the shared library's handle if successful, throws java/lang/UnsatisfiedLinkError on failure
*
* DLL: jvm
*/
/* NOTE this is required by JDK15+ jdk.internal.loader.NativeLibraries.load().
* it is only invoked by jdk.internal.loader.BootLoader.loadLibrary().
*/
void* JNICALL
JVM_LoadLibrary(const char *libName)

@pshipton

Copy link
Copy Markdown
Member

How does it work on Hotspot? Maybe the JCL causes/forces a load of KeyChainStore earlier? Pls check -verbose output.

@JasonFengJ9

Copy link
Copy Markdown
Member Author

-verbose output

Attached from a standalone testcase run.
riJDK16verbose.txt

@JasonFengJ9

Copy link
Copy Markdown
Member Author

Maybe the JCL causes/forces a load of KeyChainStore earlier?

This class is not required by most tests (that's why it only shows up in a specific internal test). So I think it shouldn't tax each JVM for the cost of loading this class and its associates (yeah, this PR always loads the native library in question which is a minimum cost).

@pshipton

Copy link
Copy Markdown
Member

I think JVM_LoadLibrary() should check the calling ClassLoader rather than having to preload libraries.

@pshipton

Copy link
Copy Markdown
Member

Even if JVM_LoadLibrary() is only called from BootLoader.loadLibrary(), we can check the caller of BootLoader.loadLibrary().

@JasonFengJ9

JasonFengJ9 commented Jun 16, 2021 •

Copy link
Copy Markdown
Member Author

Even if JVM_LoadLibrary() is only called from BootLoader.loadLibrary(), we can check the caller of BootLoader.loadLibrary().

Assuming here check if the caller classloader is BootLoader, I am guessing the proposal is to register the library being loading as a bootstrap library if it is a BootLoader.
However JDK16+ introduced JNI, and raw native libraries in which JNI_OnLoad and JNI_OnUnload have to be ignored, in addition, a native library could be unloaded (from API perspective) hence not all libraries loaded by BootLoader can be registered as bootstrap.

@gacholio

Copy link
Copy Markdown
Contributor

@pshipton Have your concerns been addressed?

@pshipton

Copy link
Copy Markdown
Member

Have your concerns been addressed?

No. This change is a hack, preloading libosxsecurity.dylib when this should not be necessary. I expect the OpenJ9 implementation for loading libraries in jdk16+ is wrong somehow. I believe Jason is investigating. I'll set this as draft for now.

@pshipton
pshipton marked this pull request as draft June 22, 2021 18:14
@JasonFengJ9

Copy link
Copy Markdown
Member Author

I believe Jason is investigating.

This is correct that I requested permission to investigate RI JDK16+ native library loading mechanism, and hope to discover a better fix.
On the other hand, I would like to make following notes:

  1. The permission request is still pending which means the investigation hasn't started yet;
  2. At this moment I don't have ETA for a better solution.

Alternatively I think this PR can serve as an intermediate fix for the internal test failure which technically is required for JDK16 delivery, and pursue the RI way for next release instead.

@pshipton

Copy link
Copy Markdown
Member

I don't see a reason we need to fix the internal test immediately, I'd prefer to wait for the proper fix.

@JasonFengJ9

Copy link
Copy Markdown
Member Author

Update:
Access permission was approved on Aug. 11th, investigating but no significant progress yet.

@JasonFengJ9 JasonFengJ9 changed the title MacOS register bootstrap library libosxsecurity.dylib JDK17+ : Try BootLoader if bootstrapClassLoader didn't find the native Aug 30, 2021
@JasonFengJ9

JasonFengJ9 commented Aug 30, 2021 •

Copy link
Copy Markdown
Member Author

Updated the PR to try BootLoader if bootstrapClassLoader didn't find the native. This solved the initial UnsatisfiedLinkError.

There are two potential future improvements (more investigation still required):

  1. Align with RI usage of jdk.internal.loader.BootLoader and jdk.internal.loader.ClassLoaders.BOOT_LOADER, and unify JDK17+ native method finding via BootLoader only;
  2. Reduce usage of registerBootstrapLibrary for JDK17+ if can't remove it completely, and adopt jdk.internal.loader.NativeLibraries loading mechanisms.

Edit: This will be backport to 0.28 if it is approved for head stream.

@JasonFengJ9
JasonFengJ9 marked this pull request as ready for review August 30, 2021 15:08
@JasonFengJ9

Copy link
Copy Markdown
Member Author

@pshipton ping

Comment thread runtime/j9vm/j9scar.tdf Resolved
Comment thread runtime/vm/vmbootlib.c Resolved
@pshipton

Copy link
Copy Markdown
Member

jenkins test sanity osx,win,zlinux jdk17

Comment thread runtime/j9vm/jvm.c Resolved
Comment thread runtime/vm/j9vm.tdf Resolved
Comment thread runtime/vm/vmbootlib.c Resolved
Try BootLoader.getNativeLibraries().find(entryName) if
bootstrapClassLoader.nativelibs.find(entryName) didn't succeed;
Add trace points;
Refactoring methods affected.

Signed-off-by: Jason Feng <fengj@ca.ibm.com>
@pshipton

Copy link
Copy Markdown
Member

jenkins compile win jdk17

@pshipton
pshipton merged commit f4e2c0d into eclipse-openj9:master Sep 13, 2021
@JasonFengJ9
JasonFengJ9 deleted the osxsecurity branch September 13, 2021 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants