Skip to content

Disable Security Manager Tests #14412

Closed
Closed

Description

@babsingh

This issue applies to tests that utilize System.setSecurityManager.

JDK18

In JDK18, the security manager is disallowed by default when the java.security.manager system property is not specified. Currently, the security manager tests in OpenJ9 cannot be selectively disabled. To keep the security manager tests correctly functioning, -Djava.security.manager=allow needs to be specified while running these tests with OpenJ9 and other implementations.

As per adoptium/TKG#263 (comment), only the OpenJ9 repo has tests which use System.setSecurityManager. The second commit in #14329 sets the java.security.manager property for the impacted tests in OpenJ9.

Related:

Future JDK

In a future JDK, the security manager will be removed. So, the security manager tests in OpenJ9 will need to be disabled.

If the compilation of the tests need to be disabled, then they would need to be separated from their current environment and run standalone. This may be challenging and will require substantially more time.

Disabling the tests during runtime may be easier. This can be achieved without separating the tests. org.junit.Assume.assumeTrue allows a test to be ignored based upon a condition. In this case, the condition would return FALSE if a security manager is not allowed based upon the JDK version and java.security.manager system property.

Impacted Tests

In OpenJ9, the impacted security manager tests are:

    Impacted Java8andUp tests:
    - JCL_TEST_Java-Security
    - JCL_TEST_Java-Lang
    - JCL_TEST_Java-Lang-Invoke
    - JCL_TEST_Java-Lang_ClassLoader
    - JCL_TEST_Java-Internals
    - JCL_TEST_IBM-VM
    - JCL_TEST_Java-Lang-Ref

    Impacted JLM_Tests:
    - JLM_Tests_IBMinternal
    - TestMemoryMXBean

    Impacted Jsr292 tests:
    - jsr292Test
    - jsr292BootstrapTest

    Impacted cmdLineTests:
    - J9security
    - proxyFieldAccess

For system tests, only the framework which launches system tests is impacted: stf.load/src/stf.load/net/adoptopenjdk/loadTest/LoadTest.java. SecurityManager.checkExit is used to prevent tests from exiting the process. Currently, there is no replacement for SecurityManager.checkExit. This issue is being tracked in https://bugs.openjdk.java.net/browse/JDK-8199704. The goal is to add an ability to intercept or disable System::exit.

How to identify classes and methods dependent on the Security Manager?

This is a useful resource: https://openjdk.java.net/jeps/411. There is a section titled: Deprecate APIs for removal. It lists the classes and methods, which will be terminally deprecated and retained. This information can be used in segregation of tests, specifically to identify classes and methods which are strongly dependent on the security manager.

Activity

  1. babsingh commented on Feb 1, 2022

    @babsingh
    ContributorAuthor
  2. babsingh commented on Feb 1, 2022

    @babsingh
    ContributorAuthor

    @pshipton when the security manager is removed, do we need to disable compilation of those tests? or disabling the tests during runtime should be sufficient?

  3. pshipton commented on Feb 1, 2022

    @pshipton
    Member

    I expect the tests won't compile on the new jdk since there won't be any java.lang.SecurityManager class available.

  4. babsingh commented on Feb 1, 2022

    @babsingh
    ContributorAuthor

    only the OpenJ9 repo has tests which use System.setSecurityManager

    The above statement is wrong. The system tests do use the Security Manager. Initially, I only looked at https://github.com/adoptium/aqa-systemtest. The system tests use code from https://github.com/adoptium/STF which utilizes the Security Manager.

    Personal build: https://hyc-runtimes-jenkins.swg-devops.com/view/OpenJ9%20-%20Personal/job/Pipeline-Build-Test-Personal/11686/

    With JDK18's java.security.manager behaviour, the following exception is thrown:

    DLT stderr Exception in thread "main" java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release
    DLT stderr 	at java.base/java.lang.System.setSecurityManager(System.java:802)
    DLT stderr 	at net.adoptopenjdk.loadTest.LoadTest.overrideSecurityManager(LoadTest.java:135)
    DLT stderr 	at net.adoptopenjdk.loadTest.LoadTest.main(LoadTest.java:114)
    

    Requested help from @Mesbah-Alam to figure the best approach to handle the below failures. Current options: either add -Djava.security.manager=allow or remove the use of the SecurityManager class.

    56 failing sanity.system test targets


    Test Name Duration Age
    2 - - DaaLoadTest_daa2_5m_0 6 sec
    4 - - TestJlmRemoteClassAuth_1 18 sec
    6 - - TestJlmRemoteNotifierProxyAuth_1 18 sec
    7 - - TestJlmRemoteThreadAuth_1 17 sec
    8 - - TestJlmRemoteThreadNoAuth_1 17 sec
    11 - - TestIBMJlmRemoteClassAuth_1 16 sec
    12 - - TestIBMJlmRemoteMemoryAuth_1 16 sec
    13 - - TestIBMJlmRemoteMemoryNoAuth_0 17 sec
    14 - - TestIBMJlmRemoteMemoryNoAuth_1 6.4 sec
    15 - - LambdaLoadTest_J9_5m_0 5.4 sec
    16 - - LambdaLoadTest_CS_5m_0 4.8 sec
    17 - - ParallelStreamsLoadTest_J9_0 5.3 sec
    18 - - MathLoadTest_autosimd_5m_0 4.5 sec
    19 - - MathLoadTest_autosimd_5m_1 5.5 sec
    20 - - MathLoadTest_bigdecimal_5m_0 4.6 sec
    22 - - MathLoadTest_bigdecimal_CS_5m_0 4.9 sec
    23 - - MauveSingleInvocLoad_J9_5m_0 8.3 sec
    24 - - MauveSingleInvocLoad_J9_5m_1 6.7 sec
    55 - - CLLoad_0 4.5 sec
    2 - - DaaLoadTest_daa1_5m_0 4.5 sec
    3 - - DaaLoadTest_daa2_5m_1 3.9 sec
    4 - - DaaLoadTest_daa3_5m_0 4.7 sec
    5 - - DaaLoadTest_daa3_CS_5m_0 4 sec
    6 - - TestJlmRemoteClassNoAuth_1 6.9 sec
    7 - - TestJlmRemoteMemoryAuth_1 16 sec
    9 - - TestJlmRemoteMemoryNoAuth_1 16 sec
    10 - - TestJlmRemoteNotifierProxyAuth_0 15 sec
    12 - - TestJlmRemoteThreadNoAuth_0 5.7 sec
    13 - - TestIBMJlmRemoteClassNoAuth_1 15 sec
    14 - - TestIBMJlmRemoteMemoryAuth_0 16 sec
    15 - - LambdaLoadTest_J9_5m_1 3.8 sec
    16 - - ParallelStreamsLoadTest_J9_1 3.7 sec
    18 - - MathLoadTest_bigdecimal_5m_1 4.5 sec
    19 - - MauveSingleThrdLoad_J9_5m_1 5.9 sec
    20 - - MauveMultiThrdLoad_5m_0 5.3 sec
    63 - - ClassLoadingTest_5m_1 4.5 sec
    65 - - NioLoadTest_5m_1 4.7 sec
    2 - - DaaLoadTest_daa1_5m_1 4.4 sec
    3 - - DaaLoadTest_daa3_5m_1 4.4 sec
    4 - - DaaLoadTest_daa1_CS_5m_0 4.9 sec
    5 - - DaaLoadTest_daa2_CS_5m_0 5.6 sec
    8 - - TestJlmRemoteClassAuth_0 16 sec
    10 - - TestJlmRemoteClassNoAuth_0 6.1 sec
    12 - - TestJlmRemoteMemoryAuth_0 16 sec
    13 - - TestJlmRemoteMemoryNoAuth_0 6.2 sec
    15 - - TestJlmRemoteThreadAuth_0 16 sec
    17 - - TestIBMJlmRemoteClassAuth_0 17 sec
    18 - - TestIBMJlmRemoteClassNoAuth_0 17 sec
    19 - - ParallelStreamsLoadTest_CS_0 4.9 sec
    20 - - MathLoadTest_autosimd_CS_5m_0 4.8 sec
    21 - - MauveSingleThrdLoad_J9_5m_0 8.3 sec
    22 - - MauveMultiThrdLoad_5m_1 7.7 sec
    59 - - CLLoad_1 5.7 sec
    64 - - ClassLoadingTest_5m_0 4.4 sec
    65 - - NioLoadTest_5m_0 4.4 sec
    67 - - ClassLoadingTest_CS_5m_0 4.9 sec

    29 failing extended.system test targets


    Test Name Duration Age
    2 - - DaaLoadTest_all_CS_5m_0 4.3 sec
    3 - - MiniMix_5m_0 5.3 sec
    5 - - MiniMix_aot_5m_0 8.5 sec
    6 - - ConcurrentLoadTest_5m_1 4.5 sec
    7 - - DBBLoadTest_5m_1 4.7 sec
    9 - - LockingLoadTest_1 4.7 sec
    11 - - UtilLoadTest_5m_1 4.6 sec
    13 - - HCRLateAttachWorkload_previewEnabled_1 5.3 sec
    16 - - SharedClassesWorkload_0 5.1 sec
    2 - - DaaLoadTest_all_5m_0 4.3 sec
    3 - - DaaLoadTest_all_5m_1 3.3 sec
    4 - - MathLoadTest_all_5m_0 3.6 sec
    5 - - MathLoadTest_all_5m_1 4.4 sec
    6 - - MathLoadTest_all_CS_5m_0 4 sec
    7 - - HCRLateAttachWorkload_previewEnabled_0 4.7 sec
    8 - - HeapHogLoadTest_5m_0 4.7 sec
    9 - - ObjectTreeLoadTest_5m_0 4.4 sec
    10 - - SharedClassesAPI_1 13 sec
    3 - - MiniMix_5m_1 7.9 sec
    4 - - ConcurrentLoadTest_5m_0 4.3 sec
    6 - - DBBLoadTest_5m_0 4.3 sec
    7 - - LangLoadTest_5m_0 4.6 sec
    8 - - LangLoadTest_5m_1 5.5 sec
    10 - - LockingLoadTest_0 4.4 sec
    11 - - UtilLoadTest_5m_0 4.4 sec
    14 - - HeapHogLoadTest_5m_1 4.5 sec
    15 - - ObjectTreeLoadTest_5m_1 4.5 sec
    17 - - SharedClassesAPI_0 14 sec
    18 - - SharedClassesWorkload_1 6.1 sec
  5. Mesbah-Alam commented on Feb 1, 2022

    @Mesbah-Alam
    Contributor
    DLT stderr 	at net.adoptopenjdk.loadTest.LoadTest.overrideSecurityManager(LoadTest.java:135)
    

    Yes, all load tests in STF based system tests will invoke LoadTests.java and thus invoke overrideSecurityManager(). From the comment in the code, it was added there to "block System.exit attempts" - that'd probably be made by the third party Mauve tests that system tests run. We need to find a replacement solution to do the same; we may add -Djava.security.manager=allow for JDK18 as a temporary solution.

  6. pshipton commented on Feb 2, 2022

    @pshipton
    Member

    For jdk18 using -Djava.security.manager=allow is the permanent solution. I'm guessing sometime between jdk19 and jdk21 (and I'm leaning towards jdk19) the java.lang.SecurityManager will be removed. -Djava.security.manager=allow won't work and code that uses SecurityManager won't compile. We should consider how we are going to handle that when it happens.

    The other one to think about is https://openjdk.java.net/jeps/421, finalization will be removed as some point as well. Not sure if any system tests are using it. I'm guessing finalization may be disabled by default in jdk19 and removed in jdk20.

  7. JasonFengJ9 commented on Feb 2, 2022

    @JasonFengJ9
    Member

    I'm guessing sometime between jdk19 and jdk21 (and I'm leaning towards jdk19) the java.lang.SecurityManager will be removed.

    If that's a correct prediction, disabling Security Manager tests (build & run) can start from JDK18 instead of setting -Djava.security.manager=allow for non-LTS versions before its actual removal. The tests for JDK17 and earlier LTS levels should be retained.

    JEP 421 (Finalization removal) will have similar requirements as well, but I think it is in much smaller scale comparing to how much security manager tests present.

  8. babsingh commented on Feb 2, 2022

    @babsingh
    ContributorAuthor

    disabling Security Manager tests (build & run) can start from JDK18 instead of setting -Djava.security.manager=allow for non-LTS versions before its actual removal

    There are OpenJDK and other non-openj9 tests related to the security manager for JDK18. Disabling security manager tests in OpenJ9 instead of setting -Djava.security.manager=allow may prevent us to pass those non-openj9 tests which are not frequently run.

    +1 to begin segregating/isolating security manager tests so they can be disabled once the security manager is removed in a future JDK. Segregating security manager tests will be challenging since it is widely used in functional and system tests. Also, isolating some tests may be difficult because they are deeply dependent on their current test environment. I don't think it is feasible to achieve this for the JDK18 release.

  9. 14 remaining items

  10. cjjdespres commented on Feb 28, 2022

    @cjjdespres
    Contributor

    @babsingh Out of curiosity, did the failures in your comment all look like

    DLT stderr Exception in thread "main" java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release
    DLT stderr 	at java.base/java.lang.System.setSecurityManager(System.java:802)
    DLT stderr 	at net.adoptopenjdk.loadTest.LoadTest.overrideSecurityManager(LoadTest.java:135)
    DLT stderr 	at net.adoptopenjdk.loadTest.LoadTest.main(LoadTest.java:114)
    

    as you mentioned? There were a lot of recent JITServer test failures with substantial overlap with that list, and I wanted to confirm whether or not that was coincidence. (None of the JITServer test failures looked like that, of course).

  11. babsingh commented on Feb 28, 2022

    @babsingh
    ContributorAuthor

    did the failures in #14412 (comment) all look like as you mentioned?

    @cjjdespres Yes, all failures had UnsupportedOperationException when System.setSecurityManager was invoked with the security manager disabled. None of the failures in #14538 are related to the security manager.

  12. theresa-m commented on Feb 4, 2025

    @theresa-m
    Contributor

    The system test frameworks reliance on the security manager has been resolved.
    @babsingh do you want this issue to remain open?

  13. babsingh commented on Feb 4, 2025

    @babsingh
    ContributorAuthor

    Yes, we can close it. #20563 tracks the removal of the security manager and the disablement of all security manager tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions