Skip to content

Introduce Off-Heap Technology for Large Arrays in Region-Based Garbage Collector #14005

Open

Description

@jonoommen

Introduce an optimized memory management for large arrays in the balanced region-based garbage collector. More specifically, this is a new heap scheme and large object organization (in-heap and off-heap), an enhanced method for allocating large objects (proxy objects), and an optimized technique of accessing large array during runtime (JIT optimizations).

This organization not only unburdens large segregated array allocation from the heap, but it also preserves the illusion to the Just-In-Time (JIT) compiler through a clever object layout that all array elements can be accessed as if all arrays were contiguous. Furthermore, we can leverage such object layout (object proxies) to optimize memory management for large objects in a heap, by introducing off-heap.

Depending on the GC policy, there may be some changes to indexable object fields/properties. Please see this Excel sheet for a summary of key information regarding this feature.

Milestones:

  • Introduce extra U_64 data field to Indexable Object Headers
  • Make sure the dataAddr field always contains the correct value, which is the address pointing right after the array header
  • Make use of the introduced extra void * data field
  • Implement a mechanism that provides a way to map sparse-heap data back to proxy objects.
  • Introduce an off-heap/secondary-heap for large array objects. This heap relies on vast, under-utilized 64-bit address space; hence, it is large enough to store any large array object without the need for compaction ever.
  • Implement a mechanism to identify all off-heap objects. Since there's no need to ever compact the off-heap, we'll introduce a mechanism that identifies large objects in the off-heap area and release them whenever they die.
  • Implement remaining functionality for the feature to allow for full GC/VM feature functionality (running WITH -Xint)
  • Implement dual indexable data address headers in order to minimize gencon regression due to the introduction of the extra void * dataAddr header field
  • Merge dual header changes into off-heap branches and fix issues introduced from this merge
  • Implement corresponding JIT off-heap functionality to allow for full feature functionality (running WITHOUT -Xint)
  • Ensure that there are no major performance impacts to GC policies that are not using off-heap technology
  • Report off-heap allocations correctly in a Javacore file

Activity

  1. amicic commented on Nov 30, 2021

    @amicic
    Contributor
  2. vijaysun-omr commented on Dec 1, 2021

    @vijaysun-omr
    Contributor

    @jonoommen there was some mention of an idea for new shape for zero-sized arrays that perhaps would be in use even for policies other than balanced GC and would move away from the complexity of having two different array length fields just for zero-sized arrays. The JIT team was interested in learning more details of such a design proposal if you can provide it so that we can assess how hard it might be to implement on the JIT side.

  3. jonoommen commented on Dec 2, 2021

    @jonoommen
    ContributorAuthor

    @vijaysun-omr yes there was mention of this however since this will certainly be a non-trivial addition to the current feature, after various discussions the plan is to table this to a later date as more of an add-on/seperate feature once off-heap has proven to be useful.

  4. dmitripivkine commented on Dec 2, 2021

    @dmitripivkine
    Contributor

    @vijaysun-omr @0xdaryl

    The followed suggestion is an optional, it is not affected off-heap work. May be we should move this discussion to separate item.

    An idea (provided by @amicic) is to use dataAddr field for recognition of Discontiguous by setting it to NULL (instead of size == 0) and have it set to non-NULL value for all kinds of Contiguous (in-heap or off-heap). It can be set not-NULL some way for 0-Length arrays too to make them be "Contiguous" technically (for example to address next after object header). In this case we can use single unified Indexable Object header
    <class/flags><size><dataAddr>
    for all types of Arrays - Contiguous or Discontiguous, in-heap or off-heap (this header is used for Contiguous now).

    This change:

    • allows to have unified way to handle all types of arrays regardless GC policy.
    • allows to eliminate special Discontiguous header (required to be handled differently as well of staying on the way to use dataAddr for recognition at first place due different offset of this field for Contiguous and Discontiguous headers)
    • speed up handling of Contiguous arrays - if it is not a 0-Length array read of pointer to array data dataAddr is required any way; if it is 0-Length array it can not be Discontiguous so either check size == 0 or dataAddr != NULL is sufficient (if necessary)
    • there is possibility for further improvement: despite we can not eliminate usage of Discontiguous at the moment (still be used in Metronome and Balanced legacy) we can set global flag at JVM startup time (and never change it during live time), for example "Contiguous Only". If this flag is set JIT can omit check for dataAddr == NULL in generated code. (Sure there are obvious limitations/checks for sharing/passing run-to-run for such generated code in case of JIT-Server or Portable AOT)

    There are few challenges on the way:

    • implement explicit new way of handling of real Discontiguous arrays across JVM code (with size > 0)
    • change logic for 0-Length array from old form <class/flags><0><0> to new one <class/flags><0><dataAddr>.
    • currently dataAddr field exists for 64-bit platforms. I am not sure what is the best - have two Object Models 32-bit/64-bit (keeping in mind that 32-bit support exists for Java 8 only) or add dataAddr slot to 32-bit platforms to to have unified way to handle objects
  5. Mesbah-Alam commented on Mar 28, 2023

    @Mesbah-Alam
    Contributor

    Does this new feature require new system tests or test modes to be enabled?

    FYI, the table here summarizes all the system tests we currently run at Adoptium. It lists modes (GC options in particular), workloads used, duration, brief description, etc. of each system test. We may refer to it to asses if existing tests will give enough coverage.

  6. vijaysun-omr commented on Mar 28, 2023

    @vijaysun-omr
    Contributor

    Thanks @Mesbah-Alam for the table. Looking at it, it seems quite skewed towards gencon, i.e. I see that we must be running balanced GC only in a small fraction of the overall test time. I feel it is probably safe to say that we need more balanced GC testing especially given the significant change to that GC policy due to this off-heap technology. I'll see what @amicic and @dmitripivkine opinions are.

  7. dmitripivkine commented on Mar 28, 2023

    @dmitripivkine
    Contributor

    Yes, I agree Balanced should get better coverage. Also, taking to consideration specific of Off-heap implementation, one of possible outcomes might be memory leak (from small to catastrophic) if implementation(s) of Virtual Memory management is not good enough for all platforms needs to be covered (platform/OS type/OS distribution etc.) I don't think is is something we are watching now. I am not sure how to address this better.

  8. Mesbah-Alam commented on Mar 30, 2023

    @Mesbah-Alam
    Contributor

    Please let us know which tests we need run with balanced GC policy, in addition to the existing ones.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions