You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Introduce Off-Heap Technology for Large Arrays in Region-Based Garbage Collector #14005
Introduce an optimized memory management for large arrays in the balanced region-based garbage collector. More specifically, this is a new heap scheme and large object organization (in-heap and off-heap), an enhanced method for allocating large objects (proxy objects), and an optimized technique of accessing large array during runtime (JIT optimizations).
This organization not only unburdens large segregated array allocation from the heap, but it also preserves the illusion to the Just-In-Time (JIT) compiler through a clever object layout that all array elements can be accessed as if all arrays were contiguous. Furthermore, we can leverage such object layout (object proxies) to optimize memory management for large objects in a heap, by introducing off-heap.
Depending on the GC policy, there may be some changes to indexable object fields/properties. Please see this Excel sheet for a summary of key information regarding this feature.
Milestones:
Introduce extra U_64 data field to Indexable Object Headers
Make sure the dataAddr field always contains the correct value, which is the address pointing right after the array header
Make use of the introduced extra void * data field
Implement a mechanism that provides a way to map sparse-heap data back to proxy objects.
Introduce an off-heap/secondary-heap for large array objects. This heap relies on vast, under-utilized 64-bit address space; hence, it is large enough to store any large array object without the need for compaction ever.
Implement a mechanism to identify all off-heap objects. Since there's no need to ever compact the off-heap, we'll introduce a mechanism that identifies large objects in the off-heap area and release them whenever they die.
Implement remaining functionality for the feature to allow for full GC/VM feature functionality (running WITH -Xint)
Implement dual indexable data address headers in order to minimize gencon regression due to the introduction of the extra void * dataAddr header field
Merge dual header changes into off-heap branches and fix issues introduced from this merge
Implement corresponding JIT off-heap functionality to allow for full feature functionality (running WITHOUT -Xint)
Ensure that there are no major performance impacts to GC policies that are not using off-heap technology
Report off-heap allocations correctly in a Javacore file
@jonoommen there was some mention of an idea for new shape for zero-sized arrays that perhaps would be in use even for policies other than balanced GC and would move away from the complexity of having two different array length fields just for zero-sized arrays. The JIT team was interested in learning more details of such a design proposal if you can provide it so that we can assess how hard it might be to implement on the JIT side.
@vijaysun-omr yes there was mention of this however since this will certainly be a non-trivial addition to the current feature, after various discussions the plan is to table this to a later date as more of an add-on/seperate feature once off-heap has proven to be useful.
The followed suggestion is an optional, it is not affected off-heap work. May be we should move this discussion to separate item.
An idea (provided by @amicic) is to use dataAddr field for recognition of Discontiguous by setting it to NULL (instead of size == 0) and have it set to non-NULL value for all kinds of Contiguous (in-heap or off-heap). It can be set not-NULL some way for 0-Length arrays too to make them be "Contiguous" technically (for example to address next after object header). In this case we can use single unified Indexable Object header <class/flags><size><dataAddr>
for all types of Arrays - Contiguous or Discontiguous, in-heap or off-heap (this header is used for Contiguous now).
This change:
allows to have unified way to handle all types of arrays regardless GC policy.
allows to eliminate special Discontiguous header (required to be handled differently as well of staying on the way to use dataAddr for recognition at first place due different offset of this field for Contiguous and Discontiguous headers)
speed up handling of Contiguous arrays - if it is not a 0-Length array read of pointer to array data dataAddr is required any way; if it is 0-Length array it can not be Discontiguous so either check size == 0 or dataAddr != NULL is sufficient (if necessary)
there is possibility for further improvement: despite we can not eliminate usage of Discontiguous at the moment (still be used in Metronome and Balanced legacy) we can set global flag at JVM startup time (and never change it during live time), for example "Contiguous Only". If this flag is set JIT can omit check for dataAddr == NULL in generated code. (Sure there are obvious limitations/checks for sharing/passing run-to-run for such generated code in case of JIT-Server or Portable AOT)
There are few challenges on the way:
implement explicit new way of handling of real Discontiguous arrays across JVM code (with size > 0)
change logic for 0-Length array from old form <class/flags><0><0> to new one <class/flags><0><dataAddr>.
currently dataAddr field exists for 64-bit platforms. I am not sure what is the best - have two Object Models 32-bit/64-bit (keeping in mind that 32-bit support exists for Java 8 only) or add dataAddr slot to 32-bit platforms to to have unified way to handle objects
Does this new feature require new system tests or test modes to be enabled?
FYI, the table here summarizes all the system tests we currently run at Adoptium. It lists modes (GC options in particular), workloads used, duration, brief description, etc. of each system test. We may refer to it to asses if existing tests will give enough coverage.
Thanks @Mesbah-Alam for the table. Looking at it, it seems quite skewed towards gencon, i.e. I see that we must be running balanced GC only in a small fraction of the overall test time. I feel it is probably safe to say that we need more balanced GC testing especially given the significant change to that GC policy due to this off-heap technology. I'll see what @amicic and @dmitripivkine opinions are.
Yes, I agree Balanced should get better coverage. Also, taking to consideration specific of Off-heap implementation, one of possible outcomes might be memory leak (from small to catastrophic) if implementation(s) of Virtual Memory management is not good enough for all platforms needs to be covered (platform/OS type/OS distribution etc.) I don't think is is something we are watching now. I am not sure how to address this better.
Activity
amicic commentedon Nov 30, 2021
FYI: @vijaysun-omr @0xdaryl @VermaSh
vijaysun-omr commentedon Dec 1, 2021
@jonoommen there was some mention of an idea for new shape for zero-sized arrays that perhaps would be in use even for policies other than balanced GC and would move away from the complexity of having two different array length fields just for zero-sized arrays. The JIT team was interested in learning more details of such a design proposal if you can provide it so that we can assess how hard it might be to implement on the JIT side.
jonoommen commentedon Dec 2, 2021
@vijaysun-omr yes there was mention of this however since this will certainly be a non-trivial addition to the current feature, after various discussions the plan is to table this to a later date as more of an add-on/seperate feature once off-heap has proven to be useful.
dmitripivkine commentedon Dec 2, 2021
@vijaysun-omr @0xdaryl
The followed suggestion is an optional, it is not affected off-heap work. May be we should move this discussion to separate item.
An idea (provided by @amicic) is to use
dataAddrfield for recognition of Discontiguous by setting it to NULL (instead ofsize == 0) and have it set to non-NULL value for all kinds of Contiguous (in-heap or off-heap). It can be set not-NULL some way for 0-Length arrays too to make them be "Contiguous" technically (for example to address next after object header). In this case we can use single unified Indexable Object header<class/flags><size><dataAddr>for all types of Arrays - Contiguous or Discontiguous, in-heap or off-heap (this header is used for Contiguous now).
This change:
dataAddrfor recognition at first place due different offset of this field for Contiguous and Discontiguous headers)dataAddris required any way; if it is 0-Length array it can not be Discontiguous so either checksize == 0ordataAddr != NULLis sufficient (if necessary)dataAddr == NULLin generated code. (Sure there are obvious limitations/checks for sharing/passing run-to-run for such generated code in case of JIT-Server or Portable AOT)There are few challenges on the way:
size > 0)<class/flags><0><0>to new one<class/flags><0><dataAddr>.dataAddrfield exists for 64-bit platforms. I am not sure what is the best - have two Object Models 32-bit/64-bit (keeping in mind that 32-bit support exists for Java 8 only) or adddataAddrslot to 32-bit platforms to to have unified way to handle objectsIntroduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Introduce Off-Heap Technology for Large Arrays
Mesbah-Alam commentedon Mar 28, 2023
Does this new feature require new system tests or test modes to be enabled?
FYI, the table here summarizes all the system tests we currently run at Adoptium. It lists modes (GC options in particular), workloads used, duration, brief description, etc. of each system test. We may refer to it to asses if existing tests will give enough coverage.
vijaysun-omr commentedon Mar 28, 2023
Thanks @Mesbah-Alam for the table. Looking at it, it seems quite skewed towards gencon, i.e. I see that we must be running balanced GC only in a small fraction of the overall test time. I feel it is probably safe to say that we need more balanced GC testing especially given the significant change to that GC policy due to this off-heap technology. I'll see what @amicic and @dmitripivkine opinions are.
dmitripivkine commentedon Mar 28, 2023
Yes, I agree Balanced should get better coverage. Also, taking to consideration specific of Off-heap implementation, one of possible outcomes might be memory leak (from small to catastrophic) if implementation(s) of Virtual Memory management is not good enough for all platforms needs to be covered (platform/OS type/OS distribution etc.) I don't think is is something we are watching now. I am not sure how to address this better.
Mesbah-Alam commentedon Mar 30, 2023
Please let us know which tests we need run with balanced GC policy, in addition to the existing ones.