Skip to content

Modifications for Off-Heap Technology for Large Arrays - #7421

Merged
0xdaryl merged 1 commit into
eclipse-omr:masterfrom
LinHu2016:off-heap-secondpart
Jul 31, 2024
Merged

0xdaryl merged 1 commit into
eclipse-omr:masterfrom
LinHu2016:off-heap-secondpart

Conversation

@LinHu2016

@LinHu2016 LinHu2016 commented Jul 22, 2024 •

Copy link
Copy Markdown
Contributor

Modifications required to complete the introduction of an optimized
memory management scheme for large arrays in a region-based
garbage collector.
More specifically, this is a new heap scheme and large
object organization (in-heap and off-heap), an
enhanced method for allocating large objects (proxy
objects), and an optimized technique of accessing
large arrays during runtime (JIT optimizations).

  • new define OMR_GC_SPARSE_HEAP_ALLOCATION
  • set OMR_GC_SPARSE_HEAP_ALLOCATION ON
  • obsolete Trc_MM_double_map_Entry, once all usages are updated.
  • replace J9VM_GC_ENABLE_SPARSE_HEAP_ALLOCATION with
    OMR_GC_SPARSE_HEAP_ALLOCATION.
  • empty method offsetOfContiguousDataAddrField() in ObjectModel
  • empty method generateArrayElementShiftAmountTrees() in TransformUtil
  • empty method isOffHeapAllocationEnabled() in ObjectModel

Signed-off-by: hulin linhu@ca.ibm.com

@LinHu2016

Copy link
Copy Markdown
Contributor Author

@amicic @dmitripivkine please review the changes, Thanks

#if defined(OMR_GC_DOUBLE_MAP_ARRAYLETS)
, isArrayletDoubleMapRequested(false)
, isArrayletDoubleMapAvailable(false)
#endif /* defined(OMR_GC_DOUBLE_MAP_ARRAYLETS) */

@dmitripivkine dmitripivkine Jul 22, 2024 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two fields are defined under the same flag. If if defined is removed here, it should be removed at fields definitions as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good observation. a preference is to keep the ifdefs if possible (if there are no spots in any current or more importantly new code that uses the variables without being wrapped with the same build flag)

Comment thread gc/base/j9mm.tdf
TraceEvent=Trc_MM_clearheap Overhead=1 Level=1 Group=gclogger Template="clearheap with free memory size: %zu, object size: %zu"
TraceEvent=Trc_MM_cleanUpSegmentsInAnonymousClassLoader_stats Overhead=1 Level=1 Group=gclogger Template="Anonymous classes: total: %zu, unloaded: %zu, searched ROM: %zu, found ROM fast: %zu, found ROM slow: %zu, not found ROM: %zu"

TraceEntry=Trc_MM_double_map_EntryNew Overhead=1 Level=3 Group=arraylet Template="MM_IndexableObjectAllocationModel::doubleMapArraylets. originalDataSize: %p, adjustedDataSize: %p, spine: %p, leafSize: %p leavesCount: %zu"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have to remember to obsolete the old one, once all (downstream) usages are updated

Comment thread gc/base/GCExtensionsBase.cpp Outdated
}

bool
MM_GCExtensionsBase::isAddressWithinHeap(void *address)

@amicic amicic Jul 23, 2024 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's try without introducing this method. instead, we can do:

  • simplest from testing perspective, but probably least desirable, introduce it on J9 side
  • use (NULL != regionManager->regionDescriptorForAddress()) instead, which probably also a reasonable simple compromise
  • examine the caller sites more carefully and see if it's really need at all, or if we can use isDataAddr(Non)Adjacent instead (requires extra non-trivial testing)

@LinHu2016
LinHu2016 force-pushed the off-heap-secondpart branch 4 times, most recently from 705879b to 9aa91c8 Compare July 23, 2024 18:38
Comment thread gc/base/GCExtensionsBase.hpp Outdated
, isArrayletDoubleMapRequested(false)
, isArrayletDoubleMapAvailable(false)
#endif /* defined(OMR_GC_DOUBLE_MAP_ARRAYLETS) */
#endif /* OMR_GC_DOUBLE_MAP_ARRAYLETS */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed 'defined()', let's restore the line to what exactly it was

@LinHu2016
LinHu2016 force-pushed the off-heap-secondpart branch 4 times, most recently from 4ea093f to 52bea29 Compare July 24, 2024 14:00
@LinHu2016
LinHu2016 force-pushed the off-heap-secondpart branch 2 times, most recently from 9441e75 to 1a8295b Compare July 24, 2024 14:58
@amicic

amicic commented Jul 24, 2024

Copy link
Copy Markdown
Contributor

@0xdaryl please, review JIT changes, or if you prefer, arrange a separate PR

@LinHu2016
LinHu2016 force-pushed the off-heap-secondpart branch from 1a8295b to 94fbd3f Compare July 30, 2024 19:39
Modifications required to complete the introduction of an optimized
memory management scheme for large arrays in a region-based garbage
collector.
More specifically, this is a new heap scheme and large
object organization (in-heap and off-heap), an
enhanced method for allocating large objects (proxy
objects), and an optimized technique of accessing
large arrays during runtime (JIT optimizations).

 - new define OMR_GC_SPARSE_HEAP_ALLOCATION
 - set OMR_GC_SPARSE_HEAP_ALLOCATION ON
 - obsolete Trc_MM_double_map_Entry, once all usages are updated.
 - replace J9VM_GC_ENABLE_SPARSE_HEAP_ALLOCATION with
   OMR_GC_SPARSE_HEAP_ALLOCATION.
 - empty method offsetOfContiguousDataAddrField() in ObjectModel
 - empty method generateArrayElementShiftAmountTrees() in TransformUtil
 - empty method isOffHeapAllocationEnabled() in ObjectModel

Signed-off-by: hulin <linhu@ca.ibm.com>
@LinHu2016
LinHu2016 force-pushed the off-heap-secondpart branch from 94fbd3f to 982a33b Compare July 31, 2024 19:12
@0xdaryl 0xdaryl self-assigned this Jul 31, 2024
@0xdaryl

0xdaryl commented Jul 31, 2024

Copy link
Copy Markdown
Contributor

Jenkins build all

@0xdaryl

0xdaryl commented Jul 31, 2024

Copy link
Copy Markdown
Contributor

@LinHu2016 : I see you've pushed recently. Does this require a GC re-review or are you just addressing Alek's comments?

@amicic

amicic commented Jul 31, 2024

Copy link
Copy Markdown
Contributor

No further changes are required.
I believe, the latest push was just a rebase to align with some changes from J9 repo for testing purposes (of J9).

@0xdaryl
0xdaryl merged commit cc9645b into eclipse-omr:master Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants