Skip to content

Adding Concurrent Mark timing stats and extra heap resizing flags - #5825

Merged
0xdaryl merged 1 commit into
eclipse-omr:masterfrom
cedrichansen:heapHeuristics
Sep 21, 2021
Merged

0xdaryl merged 1 commit into
eclipse-omr:masterfrom
cedrichansen:heapHeuristics

Conversation

@cedrichansen

@cedrichansen cedrichansen commented Feb 24, 2021 •

Copy link
Copy Markdown
Contributor

This change introduces several flags which are used to keep track of Concurrent marking work timing/cost.
This change also introduces a handful of flags which can be used by the GC to improve heap resizing.

  • _concurrentMarkProcessStartTime keeps track of the cpu time for a concurrent mark increment
  • Added additional flags for heap expansion and contraction.
  • Added MM_HeapSizingData. This struct keeps track of several key characteristics that can be used by a GC to resize the heap
  • changed heapExpansionGCTimeThreshold and heapContractionGCTimeThreshold to MM_UserSpecifiedParameterUDATA. Renamed the fields to heapExpansionGCRatioThreshold and heapContractionGCRatioThreshold respectively. Doing this allows each GC policy to configure their own defaults.
  • dnssExpectedTimeRatioMaximum and dnssExpectedTimeRatioMinimum has been changed in the same way as described for heapExpansionGCTimeThreshold
  • Introduction of tarokTargetMaxPauseTime. This will serve as the default pause time target for certain GC policies in which a target pause time will be used
  • reportHeapResizeAttempt() now takes a memoryType param. This allows GC policies which are resizing logical spaces to print a heap-resize line for those logical spaces
  • Introduced several tracepoints

Mostly unrelated to the main work in this PR, the following utility function has been added

  • MM_Math::weightedAverage() for doubles has been added

This PR does not contain any functional/behavioural changes to the GC

Finally, a follow up PR should be performed, which does the following:

  • Trc_MM_MemorySubSpaceTarok_calculateExpandSize_Exit1 should be made obsolete
  • Trc_MM_MemorySubSpaceTarok_calculateTargetContractSize_Entry should be made obsolete
  • dnssExpectedTimeRatioMaximum/Minimum should be removed, while dnssExpectedRatioMaximum/Minimum should be kept
  • heapExpansionGCTimeThreshold and heapContractionGCTimeThreshold should be removed, and heapExpansionGCRatioThreshold and heapContractionGCRatioThreshold should be kept.

Signed-off-by: Cedric Hansen cedric.hansen@ibm.com

@cedrichansen

Copy link
Copy Markdown
Contributor Author

@amicic FYI

Comment thread gc/base/Configuration.cpp Outdated
extensions->dnssExpectedTimeRatioMinimum._valueSpecified = DEFAULT_LOCAL_GC_TIME_RATIO_FOR_NURSERY_CONTRACT;
}


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we somehow set here some generic values with special meaning 'uninitialized' like 0.00 (or even do nothing) and then set Gencon specific ones in ConfigurationGenerational ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When dnssExpectedTimeRatioMinimum is initialized in MM_GCExtensionsBase, the default value for _valueSpecified is indeed 0.0. Is that what you meant for the uninitialized value?

As for setting the default in ConfigurationGenerational, I think it can be done - I will look into it

@cedrichansen cedrichansen Feb 24, 2021 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see MM_ConfigurationGenerational::initialize() for new location of this initialization logic as of latest update

Comment thread gc/base/gcutils.cpp Outdated
case SCAV_RATIO_TOO_HIGH:
return "excessive time being spent scavenging";
case GC_HIGH_SPACE_LOW:
return "space too low and gc ratio too high";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align the names of those too opposite constants. For example:

FREE_SPACE_HIGH/LOW_AND_GC_TIME_LOW/HIGH

I wonder if AND or OR is more appropriate!?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think given their usage, it might make more sense to use "OR" in those 2 cases, since "AND" will not always be the case.

ie, gc ratio might be too high, but free space could be fine, suggesting expansion

@cedrichansen
cedrichansen force-pushed the heapHeuristics branch 2 times, most recently from f182369 to 808a43a Compare February 24, 2021 21:47
@cedrichansen
cedrichansen force-pushed the heapHeuristics branch 2 times, most recently from 707a84e to f9a1e0d Compare March 15, 2021 16:10
cedrichansen added a commit to cedrichansen/openj9 that referenced this pull request Mar 15, 2021
This change introduces 2 different major componants, which work together.

---------Change 1: New heap sizing logic in balanced gc
The old heap sizing logic, relied primarily on free memory %, in relation to -Xminf and -Xmaxf to decide whether or not the heap should shrink/expand.
If free memory was within the acceptable boundary, the heap sizing logic would then look at gc cpu% to decide whether to expand/contract.

The new logic (which is included in this commit), aims to take a more hybrid approach, where TENURE free memory % and gc cpu % are working together, and weighed equally, to determine whether or not the heap needs to change.
The function ```calculateCurrentHybridHeapOverhead``` determines what the current "hybrid overhead" of the heap/gc is (blend of gc cpu % and free memory %).
If this "hybrid overhead" is above -Xmaxt (default to 13%), the heap will expand. Conversely, if the "hybrid overhead" is below -Xmint (5% default), the heap will shrink.

--------Change 2: New eden sizing logic in balanced

The current logic for eden sizing is fairly simple - By default, eden is taken to be 25% of the heap - unless specified otherwise by -Xmn/-Xmns/-Xmnx

The new logic, aims to make use of a few heuristics to improve overall PGC overheaad (% of time being active relative to total time), while trying to respect the newly introduced, `tarokTargetMaxPauseTime`.
Eden sizing logic now adheres to the following high level concepts:

 1. If the heap is not fully expanded ( that is current heap size <= -Xsoftmx/-Xmx), then eden will increase by 10% if PGC overhead and pgc avg time blend, is above
   dnssExpectedTimeRatioMinimum (7% default). Conversely, eden will shrink by 10% if the hybrid blend of pgc overhead and pgc time, falls below dnssExpectedTimeRatioMaximum (2% default).

 2. If the heap is fully expanded, a prediction will be made as to which eden size will produce the lowest total GC cpu overhead, while attempting to respect tarokTargetMaxPauseTime, given the constraints imposed by the heap.
    This prediction accounts for PGC average time/intervals, GMP time/intervals, free Tenure space, and survival ratio of objects being copied out of eden, in order to determine the "best" eden size.

Note: The following command line options are now supported by balanced gc policy as part of this change.
	- `-Xgc:dnssExpectedTimeRatioMinimum` -> sets min expected % of time spent in pgc pauses (default 2% for balanced)
	- `-Xgc:dnssExpectedTimeRatioMaximum` -> sets max expected % of time spent in pgc pauses (default 7% for balanced)
	- `-Xgc:targetPausetime` -> sets target (tarokTargetMaxPauseTime defaults to 200ms in balanced)

Depends on: eclipse-omr/omr#5825

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
@0xdaryl

0xdaryl commented Mar 22, 2021

Copy link
Copy Markdown
Contributor

This PR needs to be rebased.

@0xdaryl

0xdaryl commented Apr 9, 2021

Copy link
Copy Markdown
Contributor

Is this PR ready for final review and CI testing?

cedrichansen added a commit to cedrichansen/openj9 that referenced this pull request Jul 14, 2021
This change introduces 2 different major componants, which work together.

---------Change 1: New heap sizing logic in balanced gc
The old heap sizing logic, relied primarily on free memory %, in relation to -Xminf and -Xmaxf to decide whether or not the heap should shrink/expand.
If free memory was within the acceptable boundary, the heap sizing logic would then look at gc cpu% to decide whether to expand/contract.

The new logic (which is included in this commit), aims to take a more hybrid approach, where TENURE free memory % and gc cpu % are working together, and weighed equally, to determine whether or not the heap needs to change.
The function ```calculateCurrentHybridHeapOverhead``` determines what the current "hybrid overhead" of the heap/gc is (blend of gc cpu % and free memory %).
If this "hybrid overhead" is above -Xmaxt (default to 13%), the heap will expand. Conversely, if the "hybrid overhead" is below -Xmint (5% default), the heap will shrink.

--------Change 2: New eden sizing logic in balanced

The current logic for eden sizing is fairly simple - By default, eden is taken to be 25% of the heap - unless specified otherwise by -Xmn/-Xmns/-Xmnx

The new logic, aims to make use of a few heuristics to improve overall PGC overheaad (% of time being active relative to total time), while trying to respect the newly introduced, `tarokTargetMaxPauseTime`.
Eden sizing logic now adheres to the following high level concepts:

 1. If the heap is not fully expanded ( that is current heap size <= -Xsoftmx/-Xmx), then eden will increase by 10% if PGC overhead and pgc avg time blend, is above
   dnssExpectedTimeRatioMinimum (7% default). Conversely, eden will shrink by 10% if the hybrid blend of pgc overhead and pgc time, falls below dnssExpectedTimeRatioMaximum (2% default).

 2. If the heap is fully expanded, a prediction will be made as to which eden size will produce the lowest total GC cpu overhead, while attempting to respect tarokTargetMaxPauseTime, given the constraints imposed by the heap.
    This prediction accounts for PGC average time/intervals, GMP time/intervals, free Tenure space, and survival ratio of objects being copied out of eden, in order to determine the "best" eden size.

Note: The following command line options are now supported by balanced gc policy as part of this change.
	- `-Xgc:dnssExpectedTimeRatioMinimum` -> sets min expected % of time spent in pgc pauses (default 2% for balanced)
	- `-Xgc:dnssExpectedTimeRatioMaximum` -> sets max expected % of time spent in pgc pauses (default 7% for balanced)
	- `-Xgc:targetPausetime` -> sets target (tarokTargetMaxPauseTime defaults to 200ms in balanced)

Depends on: eclipse-omr/omr#5825

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
@cedrichansen

cedrichansen commented Jul 14, 2021 •

Copy link
Copy Markdown
Contributor Author

@amicic PR feedback has been addressed, ready for re-review

Comment thread gc/base/GCExtensionsBase.cpp Outdated
heapContractionGCRatioThreshold._valueSpecified = 5;

heapExpansionGCTimeThreshold = 13; //To be removed
heapContractionGCTimeThreshold = 5; //To be removed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's initialized in constructor (which you will remove later), so this is not necessary

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, removing these 2 lines now

Comment thread gc/base/j9mm.tdf Outdated
TraceExit=Trc_MM_AllocationContextBalanced_acquireMPAOLRegionFromNode_Exit Overhead=1 Level=1 Group=tarok Template="MM_AllocationContextBalanced::acquireMPAOLRegionFromNode result=%p"
TraceEntry=Trc_MM_MemorySubSpaceTarok_calculateTargetContractSize_Entry2 Overhead=1 Level=1 Group=resize Template="MM_MemorySubSpaceTarok_calculateTargetContractSize Entry Allocation size = %zu"
TraceExit=Trc_MM_MemorySubSpaceTarok_calculateExpandSize_Exit2 Overhead=1 Level=1 Group=resize Template="MM_MemorySubSpaceTarok_calculateExpandSize Exit1 returning expand size of %zu bytes"
TraceEvent=Trc_MM_MemorySubSpaceTarok_calculateHybridHeapOverhead Overhead=1 Level=1 Group=resize Template="MM_MemorySubSpaceTarok_calculateHybridHeapOverhead GC componant = %f Memory Componant = %f"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spelling: Componant

Comment thread gc/base/j9mm.tdf Outdated

TraceEvent=Trc_MM_IncrementalGenerationalGC_calculateConcurrentMarkWorkTime Overhead=1 Level=2 Group=resize Template="IncrementalGenerationalGC_calculateConcurrentMarkWorkTime Concurrent GMP ratio %f previous increment concurrent worktime %llu Total concurrent worktime %llu"

TraceEvent=Trc_MM_SweepEndBalancedGC Overhead=1 Level=1 Group=gclogger Template="Sweep end. Duration %llu us" No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github diff is telling me there is missing end-of-line

@cedrichansen cedrichansen Jul 28, 2021 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix that up

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved as of latest commit

@cedrichansen
cedrichansen force-pushed the heapHeuristics branch 3 times, most recently from 193710f to 164e137 Compare September 13, 2021 19:37
…additional flags for heap resizing

This change introduces several flags which are used to keep track of Concurrent marking work timing/cost.
This change also introduces a handful of flags which can be used by the GC to improve heap resizing.

- `_concurrentMarkProcessStartTime` keeps track of the cpu time for a concurrent mark increment
- Added additional flags for heap expansion and contraction.
- Added `MM_HeapSizingData`. This struct keeps track of several key characteristics that can be used by a GC to resize the heap
- changed `heapExpansionGCTimeThreshold` and `heapContractionGCTimeThreshold` to `MM_UserSpecifiedParameterUDATA`. Renamed the fields to `heapExpansionGCRatioThreshold` and `heapContractionGCRatioThreshold` respectively. Doing this allows each GC policy to configure their own defaults.
- `dnssExpectedTimeRatioMaximum` and `dnssExpectedTimeRatioMinimum` has been changed in the same way as described for `heapExpansionGCTimeThreshold`
- Introduction of `tarokTargetMaxPauseTime`. This will serve as the default pause time target for certain GC policies in which a target pause time will be used
- `reportHeapResizeAttempt()` now takes a memoryType param. This allows GC policies which are resizing logical spaces to print a `heap-resize` line for those logical spaces
- Introduced several tracepoints

Mostly unrelated to the main work in this PR, the following utility function has been added
- `MM_Math::weightedAverage()` for doubles has been added

This PR does not contain any functional/behavioural changes to the GC

Finally, a follow up PR should be performed, which does the following:
- `Trc_MM_MemorySubSpaceTarok_calculateExpandSize_Exit1` should be made obsolete
- `Trc_MM_MemorySubSpaceTarok_calculateTargetContractSize_Entry` should be made obsolete
- `dnssExpectedTimeRatioMaximum/Minimum` should be removed, while `dnssExpectedRatioMaximum/Minimum` should be kept
- `heapExpansionGCTimeThreshold` and `heapContractionGCTimeThreshold` should be removed, and `heapExpansionGCRatioThreshold` and `heapContractionGCRatioThreshold` should be kept.

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
@amicic

amicic commented Sep 14, 2021

Copy link
Copy Markdown
Contributor

Is this PR ready for final review and CI testing?

@0xdaryl It finally is.

@0xdaryl

0xdaryl commented Sep 14, 2021

Copy link
Copy Markdown
Contributor

Jenkins build all

@0xdaryl 0xdaryl self-assigned this Sep 14, 2021
@0xdaryl

0xdaryl commented Sep 14, 2021

Copy link
Copy Markdown
Contributor

Jenkins build aarch64

@0xdaryl
0xdaryl merged commit 859de4c into eclipse-omr:master Sep 21, 2021
cedrichansen added a commit to cedrichansen/openj9 that referenced this pull request Sep 22, 2021
This change introduces 2 different major componants, which work together.

An in depth overview of the inner workings of these changes can be found eclipse-openj9#12054

---------Change 1: New heap sizing logic in balanced gc
The old heap sizing logic, relied primarily on free memory %, in relation to -Xminf and -Xmaxf to decide whether or not the heap should shrink/expand.
If free memory was within the acceptable boundary, the heap sizing logic would then look at gc cpu% to decide whether to expand/contract.

The new logic (which is included in this commit), aims to take a more hybrid approach, where TENURE free memory % and gc cpu % are working together, and weighed equally, to determine whether or not the heap needs to change.
The function `calculateCurrentHybridHeapOverhead()` determines what the current "hybrid overhead" of the heap/gc is (blend of gc cpu % and free memory %).
If this "hybrid overhead" is above -Xmaxt (default to 13%), the heap will expand. Conversely, if the "hybrid overhead" is below -Xmint (5% default), the heap will shrink.

--------Change 2: New eden sizing logic in balanced

The current logic for eden sizing is fairly simple - By default, eden is taken to be 25% of the heap - unless specified otherwise by -Xmn/-Xmns/-Xmnx

The new logic, aims to make use of a few heuristics to improve overall PGC overheaad (% of time being active relative to total time), while trying to respect the newly introduced, `tarokTargetMaxPauseTime`.
Eden sizing logic now adheres to the following high level concepts:

 1. If the heap is not fully expanded ( that is, the current heap size <= -Xsoftmx/-Xmx), then eden will increase by 5% if PGC overhead and pgc avg time blend, is above
   dnssExpectedTimeRatioMinimum (5% default). Conversely, eden will shrink by 5% if the hybrid blend of pgc overhead and pgc time, falls below dnssExpectedTimeRatioMaximum (2% default).

 2. If the heap is fully expanded, a prediction will be made as to which eden size will produce the lowest total GC cpu overhead, while attempting to respect `tarokTargetMaxPauseTime`, given the constraints imposed by the heap.
    This prediction accounts for PGC average time/intervals, GMP time/intervals, free Tenure space, and survival ratio of objects being copied out of eden, in order to determine the "best" eden size.

 3. If `tarokTargetPauseTime` is not being fully respected, then the eden sizing logic will balance respecting the % of time spent in GC goals, and target pause time goals. The actual mechanism which causes eden size to respect this value, is baked into the logic for 1. and 2. above. This value is a SOFT pause time target, which is defaulted at 200ms

Note: The following command line options are now supported by balanced gc policy as part of this change.
	- `-Xgc:dnssExpectedTimeRatioMinimum` -> sets min expected % of time spent in pgc pauses (default 2% for balanced)
	- `-Xgc:dnssExpectedTimeRatioMaximum` -> sets max expected % of time spent in pgc pauses (default 5% for balanced)
	- `-Xgc:targetPausetime` -> sets target (tarokTargetMaxPauseTime defaults to 200ms in balanced)

Depends on: eclipse-omr/omr#5825

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
cedrichansen added a commit to cedrichansen/omr that referenced this pull request Sep 22, 2021
This commit contains the follow up items from eclipse-omr#5825

This includes:
- making `Trc_MM_MemorySubSpaceTarok_calculateExpandSize_Exit1` and `Trc_MM_MemorySubSpaceTarok_calculateTargetContractSize_Entry` obsolete
- `dnssExpectedTimeRatioMaximum` and `dnssExpectedTimeRatioMinimum` were removed, since they are no longer used. The newer version (dnssExpectedRatioMaximum/Minimum) are more flexible for different GC policies
- `heapExpansionGCTimeThreshold` and `heapContractionGCTimeThreshold` were removed, since they are no longer used. The newer versions (`heapExpansionGCRatioThreshold` and `heapContractionGCRatioThreshold`) are more flexible for different GC policies

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
@cedrichansen cedrichansen mentioned this pull request Sep 22, 2021
cedrichansen added a commit to cedrichansen/omr that referenced this pull request Sep 23, 2021
This commit contains the follow up items from eclipse-omr#5825

This includes:
- making `Trc_MM_MemorySubSpaceTarok_calculateExpandSize_Exit1` and `Trc_MM_MemorySubSpaceTarok_calculateTargetContractSize_Entry` obsolete
- `dnssExpectedTimeRatioMaximum` and `dnssExpectedTimeRatioMinimum` were removed, since they are no longer used. The newer version (dnssExpectedRatioMaximum/Minimum) are more flexible for different GC policies
- `heapExpansionGCTimeThreshold` and `heapContractionGCTimeThreshold` were removed, since they are no longer used. The newer versions (`heapExpansionGCRatioThreshold` and `heapContractionGCRatioThreshold`) are more flexible for different GC policies

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
qasimy123 pushed a commit to qasimy123/openj9-jit-debug-agent that referenced this pull request Nov 26, 2021
This change introduces 2 different major componants, which work together.

An in depth overview of the inner workings of these changes can be found eclipse-openj9/openj9#12054

---------Change 1: New heap sizing logic in balanced gc
The old heap sizing logic, relied primarily on free memory %, in relation to -Xminf and -Xmaxf to decide whether or not the heap should shrink/expand.
If free memory was within the acceptable boundary, the heap sizing logic would then look at gc cpu% to decide whether to expand/contract.

The new logic (which is included in this commit), aims to take a more hybrid approach, where TENURE free memory % and gc cpu % are working together, and weighed equally, to determine whether or not the heap needs to change.
The function `calculateCurrentHybridHeapOverhead()` determines what the current "hybrid overhead" of the heap/gc is (blend of gc cpu % and free memory %).
If this "hybrid overhead" is above -Xmaxt (default to 13%), the heap will expand. Conversely, if the "hybrid overhead" is below -Xmint (5% default), the heap will shrink.

--------Change 2: New eden sizing logic in balanced

The current logic for eden sizing is fairly simple - By default, eden is taken to be 25% of the heap - unless specified otherwise by -Xmn/-Xmns/-Xmnx

The new logic, aims to make use of a few heuristics to improve overall PGC overheaad (% of time being active relative to total time), while trying to respect the newly introduced, `tarokTargetMaxPauseTime`.
Eden sizing logic now adheres to the following high level concepts:

 1. If the heap is not fully expanded ( that is, the current heap size <= -Xsoftmx/-Xmx), then eden will increase by 5% if PGC overhead and pgc avg time blend, is above
   dnssExpectedTimeRatioMinimum (5% default). Conversely, eden will shrink by 5% if the hybrid blend of pgc overhead and pgc time, falls below dnssExpectedTimeRatioMaximum (2% default).

 2. If the heap is fully expanded, a prediction will be made as to which eden size will produce the lowest total GC cpu overhead, while attempting to respect `tarokTargetMaxPauseTime`, given the constraints imposed by the heap.
    This prediction accounts for PGC average time/intervals, GMP time/intervals, free Tenure space, and survival ratio of objects being copied out of eden, in order to determine the "best" eden size.

 3. If `tarokTargetPauseTime` is not being fully respected, then the eden sizing logic will balance respecting the % of time spent in GC goals, and target pause time goals. The actual mechanism which causes eden size to respect this value, is baked into the logic for 1. and 2. above. This value is a SOFT pause time target, which is defaulted at 200ms

Note: The following command line options are now supported by balanced gc policy as part of this change.
	- `-Xgc:dnssExpectedTimeRatioMinimum` -> sets min expected % of time spent in pgc pauses (default 2% for balanced)
	- `-Xgc:dnssExpectedTimeRatioMaximum` -> sets max expected % of time spent in pgc pauses (default 5% for balanced)
	- `-Xgc:targetPausetime` -> sets target (tarokTargetMaxPauseTime defaults to 200ms in balanced)

Depends on: eclipse-omr/omr#5825

Signed-off-by: Cedric Hansen <cedric.hansen@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants