Skip to content

Many extended.openjdk tests timing out on aarch64 #2002

Closed
adoptium/aqa-tests
#2345

Description

@lumpfish

There are many tests timing out in the extended.openjdk target with messages like:

06:58:21  TEST RESULT: Error. Program `/home/jenkins/workspace/Test_openjdk16_hs_extended.openjdk_aarch64_linux/openjdkbinary/j2sdk-image/bin/java' timed out (timeout set to 960000ms, elapsed time including timeout handling was 1006476ms).

Example job log: https://ci.adoptopenjdk.net/job/Test_openjdk16_hs_extended.openjdk_aarch64_linux/7/consoleFull

The tests do not time out on other platforms - could the resources allocated to the test docker images be a factor?

Activity

  1. sxa commented on Mar 4, 2021

    @sxa
    Member

    Re-running at https://ci.adoptopenjdk.net/job/Grinder/7435 on the same system just now while the host system should otherwise be idle.

  2. self-assigned this
    on Mar 4, 2021
  3. added this to the March 2021 milestone on Mar 4, 2021
  4. sxa commented on Mar 8, 2021

    @sxa
    Member

    @smlambert Are the extended openjdk suites similar to the system test suites in that they can chew up all the CPU cycles available to them?

  5. sxa commented on Mar 9, 2021

    @sxa
    Member

    After running a few other Grinders I'm now running jdk_time_1 with 10 iterations on two affected machines (It's not 100% consistent):

  6. sxa commented on Mar 9, 2021

    @sxa
    Member

    Current theory: The docker containers are restricted, however the tests are trying to spawn significantly more processes than there are CPUs available to the container. This is causing it to reach the memory limits of the container and slow things down. It would be good to know how the tests (jdk_time_1 being a good example) decide how much processes to spawn off. At the moment on my container restricted to 8 CPUs I have 29 javac process and 21 jstack ones

  7. sxa commented on Mar 9, 2021

    @sxa
    Member

    ping @smlambert Is this expected? How many cores would you expect these tests to use up generally?

  8. smlambert commented on Mar 9, 2021

    @smlambert
    Contributor

    A while back, I believe @jerboaa added this PR adoptium/aqa-tests#1427 which may be relevant to the question 'how many cores'?

  9. sxa commented on Mar 9, 2021

    @sxa
    Member

    A while back, I believe @jerboaa added this PR AdoptOpenJDK/openjdk-tests#1427 which may be relevant to the question 'how many cores'?

    That would explain it :-) I'll see what I can do ...

    Thanks for the pointer!! (Worthy of two ! I feel)

  10. sxa commented on Mar 10, 2021

    @sxa
    Member

    The above PR seems to resolve this problem based on the jdk_time-1 Grinders I've been running (Docker | Non-docker)

    I've initiated a full extended.openjdk on a docker machine and a real machine to compare but once that PR is merged I'm confident this particular issue can be closed so I'll be able to re-enable all the machines again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions