Skip to content

Add two helpers for OSR related optimizations - #3030

Merged
vijaysun-omr merged 2 commits into
eclipse-omr:masterfrom
liqunl:osrhelper
Nov 8, 2018
Merged

vijaysun-omr merged 2 commits into
eclipse-omr:masterfrom
liqunl:osrhelper

Conversation

@liqunl

@liqunl liqunl commented Oct 4, 2018 •

Copy link
Copy Markdown
Contributor
  • Add a helper named osrFearPointHelper to mark an OSR fear point, a
    place that has been optimized with runtime assumptions and requires
    protection of OSR points. This helper helps to locate actual fear
    points, which as part of the program can be optimized, and allows the
    separation of OSR dependent optimization and OSR guard insertion.

  • Add a helper named potentialOSRPointHelper to serve as a marker of OSR
    points. Currently, OSR points consist of calls, asyncchecks and
    monitor enters. As part of the program, they can be optimized and no
    longer be identified as OSR points. However, places where they reside
    may still be OSR possible. This helper works as a bookkeeping tool to
    track place that are OSR possible.

  • Add a markdown file that covers a basic introduction to OSR.

  • Fix some node APIs such that doNotProfile is set only for nodes
    created during ilgen. doNotProfile on a bytecode info is looked at by
    OSR APIs to determine if a node is created in ilgen. Only nodes
    created in ilgen will have the OSR bookkeeping and are able to
    support OSR transition. Thus, nodes created after ILGen supposed to
    have doNotProfile set and cannot be OSR points.

Signed-off-by: liqunl liqunl@ca.ibm.com

@liqunl

liqunl commented Oct 4, 2018

Copy link
Copy Markdown
Contributor Author

@andrewcraik Can you review?

Comment thread compiler/il/OMRNode.cpp Outdated
/**
* \brief
* Return true of the node is a call with osrFearPointSymbol
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see the API documentation move to the interface in the .hpp file so that downstream projects extending the classes know how to implement the specification without reading the OMR implementation.

Comment thread compiler/optimizer/Inliner.cpp Outdated
#include "ras/DebugCounter.hpp"
#include "ras/LogTracer.hpp" // for heuristicTrace, etc
#include "runtime/Runtime.hpp"
#include "compile/SymbolReferenceTable.hpp" // for osrFearPointSymbol, etc

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the alphabetical order and move this up to the other "compile/" includes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like it is already included, i deleted this line.

Comment thread compiler/il/OMRNode.cpp Outdated
* Return true of the node is a call with osrFearPointSymbol
*/
bool
OMR::Node::isOSRFearPoint()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is only checking for calls to the special OSRFearPoint helper I think this needs a better name because this sounds very generic when this is actually very specific.

Comment thread compiler/il/OMRNode.cpp Outdated
* Return true of the node is a call with potentialOSRPointSymbol
*/
bool
OMR::Node::isPotentialOSRPoint()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is only checking for calls to the special PotentialOSRPoint helper I think this needs a better name because this sounds very generic when this is actually very specific.

}

bool
OMR::ResolvedMethodSymbol::induceOSRAfter(TR::TreeTop *insertionPoint, TR::Node* osrNode, TR::TreeTop* branch,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this new API being used? inducing an OSR only conceptually needs and insertion point and a BCI so I'm not sure what this new API is going to get us? A more detailed discussion of this API and why it would be used would be helpful. Given an induction is something you expect to happen I'm not sure why the true false etc here is helpful...

}

bool
OMR::ResolvedMethodSymbol::supportsInduceOSR(TR::Node* osrNode,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this rather than isPotentialOSRPoint or isPotentialOSRPointWithSupport?

Comment thread compiler/compile/OMRCompilation.cpp Outdated
TR::ResolvedMethodSymbol *method = bci.getCallerIndex() == -1 ?
self()->getMethodSymbol() : self()->getInlinedResolvedMethodSymbol(bci.getCallerIndex());
potentialOSRPoint = method->supportsInduceOSR(bci, tt->getEnclosingBlock(), self(), false);
potentialOSRPoint = method->supportsInduceOSR(osrNode, tt->getEnclosingBlock(), self(), false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we care about the node? We can either OSR at a BCI or not which is what this was originally asking...

@liqunl

liqunl commented Oct 11, 2018

Copy link
Copy Markdown
Contributor Author

Made some changes according to Andrew's comments. I agree that the OSR infrastructure need not know about nodes, everything should be based on bytecodes. The changes are made on the fact that the OSR infrastructure is utilizing the doNotProfile flag on bytecode info. This flag basically tells us whether a bytecode info on a node is created during ilgen. Two bytecode info objects can have the same bytecode index and caller index, but they may have different flags. The potentialOSRPointHelper call will have doNotProfile flag set to false on its bytecode info such that the ability to transition at the helper call is not affected by when the node is created.

@andrewcraik Can you continue the review process?

@liqunl liqunl changed the title Add two helpers for OSR related optimizations WIP: Add two helpers for OSR related optimizations Oct 11, 2018
@liqunl

liqunl commented Oct 11, 2018

Copy link
Copy Markdown
Contributor Author

WIP this PR since I need to rework on the alias set for the two helpers.

@liqunl liqunl changed the title WIP: Add two helpers for OSR related optimizations Add two helpers for OSR related optimizations Oct 23, 2018
@liqunl

liqunl commented Oct 23, 2018

Copy link
Copy Markdown
Contributor Author

@jdmpapin @andrewcraik May I ask for a review?

@andrewcraik

Copy link
Copy Markdown
Contributor

So there should be an assert in the creation of helper call nodes that we have not yet removed the OSR infrastructure otherwise the nodes will have been created after the OSR setup has been completed - see osrInfrastructureRemoved on OMRCompilation - this will help prevent misuse.

@andrewcraik

Copy link
Copy Markdown
Contributor

I think this change warrants a markdown file added to the docs/compiler directory explaining what the helpers are, what their bytecode indices represent and what they do since it is not obvious for someone removed from the design of this new capability.

Comment thread compiler/il/Aliases.cpp Outdated
// returns non-zero value for any call. The control flow constraint imposed by the exception
// edge will apply to all the global optimizations that may move things around. Local
// optimizations also ask `exceptionsRaised` to determine if a movement across certain point
// is dangerous.

@andrewcraik andrewcraik Oct 23, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I buy that this would work for the transition points the way they are intended to be used - that is we create a transition point targeting a bytecode index in a given method when we believe the block already has the infrastructure to do that transition. I am not sure that this will hold for a fear point. The fear point is not a point that we explicitly kept track of during ilgen and so you are relying on the block's OSR path to keep things live for a point that was not tracked during ilgen. Now if the fear points are also potential OSR points then I can see how that would work, but there is an implicit assumption on the relationship between the fear point and the potential points.

How can we be sure the exception edge for the fear point a) will exist and b) will hold all the right things live?

[Edit replaced fear with transition in the first sentence due to my own ability to type what I meant]

@jdmpapin jdmpapin Oct 23, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exception control flow is relevant for the OSR points, but not (directly) for the fear points.

Because a fear point helper call must be protected, and because it represents no computation of its own, I believe the only constraints on it have to do with this protection. If it's unprotectable, it must remain unprotectable. If OTOH it's protectable, the set of OSR points protecting it must be (roughly speaking) preserved. I say roughly speaking because any such OSR point that isn't a yield point (e.g. a potential OSR point helper call), if it could itself be protected by earlier OSR points, can be replaced with those. Protectable fear point helper calls are also allowed to become unprotectable, although that's undesirable because it would cause spurious compilation restarts.

I think these invariants could be maintained while allowing reordering of fear point helper calls w.r.t. anything other than OSR points and yield/invalidation points. These reorderings would in almost all cases be naturally disallowed simply due to the fact that the fear point is represented as a call, which even without aliasing could e.g. perform I/O.

That said, I think it's perfectly fine to add aliasing here to cause the compiler to be more conservative about reordering these helpers with other operations, so long as there is a comment to indicate that the aliasing is known to be more conservative than necessary. That would hedge against any holes in the account just above, and it can always be relaxed later if necessary.

@andrewcraik

andrewcraik commented Oct 23, 2018 •

Copy link
Copy Markdown
Contributor

There seems to be an implicit assumption that a transition point call node will be created for an existing OSR point (and it has to be for the transition to work). I would prefer some code to verify this - namely you cannot create an potential OSR point helper call and stick it in a block without the right exception setup and without the right book keeping being in place for the call. My opinion is that a preventative check like this will prevent a number of cases of potential misuse / undefined behavior later on and should make development use of the feature fail early rather than late with a more helpful error.

To facilitate these checks should the API to create these nodes require a treetop after which the call will be added so we can verify the infrastructure? We could then not provide a direct create API which would further reduce the risk for misuse?

@liqunl

liqunl commented Oct 26, 2018

Copy link
Copy Markdown
Contributor Author

@andrewcraik @jdmpapin Made some changes according to your comments.

@andrewcraik

Copy link
Copy Markdown
Contributor

@jdmpapin since I'm pretty busy can you make a first pass and I'll have a look after that?

@liqunl

liqunl commented Oct 26, 2018

Copy link
Copy Markdown
Contributor Author

Forgot to mention that the call node will have an iconst child which stores the bytecode offset of the induction target.

@jdmpapin jdmpapin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good over all 👍

I would personally prefer if we could somehow avoid creating more situations where a child node must have a particular form (i.e. must be a constant, in this case). Having mentioned that, I'll leave it to the judgment of others 🙂

Just a few very minor comments below

Comment thread compiler/il/OMRNode.cpp
// Node created outside of ilgen will have doNotProfile set, this results in OSR infrastructure believing it
// can not OSR at this node. Reset this flag since a potentialOSRPointHelper is a safe OSR transition point
//
callNode->getByteCodeInfo().setDoNotProfile(0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that this function is restricted to ilgen, does it still need to setDoNotProfile(0)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Node constructor sets doNotProfile if an originatingByteCodeNode is passed in.

Comment thread compiler/il/OMRNode.hpp Outdated

/**
* \brief
* Return true of the node is a load of static final field

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like "of" is a typo for "if", and again in the two just below

Comment thread doc/compiler/osr/OSR.md Outdated

At an OSR point, the VM state can be correctly reconstructed and is consistent with the VM state needed by the interpreter to resume the execution at a given bytecode index.

To achieve that, bookkeeping is required at all possible OSR points during ILGen. Due to the cost and complexity of bookkeeping, OSR transition is limited to some well-defined points, which are are calls, asyncchecks and monitor enters, etc.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: are are

@liqunl

liqunl commented Oct 26, 2018

Copy link
Copy Markdown
Contributor Author

@andrewcraik @jdmpapin Fixed the typo and moved the offset to the Node union. Will create an issue for the work to print the offset in the trace log.

@liqunl

liqunl commented Oct 26, 2018

Copy link
Copy Markdown
Contributor Author

Updated some comments which is not valid due to the change in the tree shape.

@andrewcraik

Copy link
Copy Markdown
Contributor

I think this is now good to go - @vijaysun-omr could you take a look?

Comment thread compiler/il/OMRNode.cpp
comp->getDebug()->newNode(self());

TR_IlGenerator * ilGen = comp->getCurrentIlGenerator();
if (ilGen)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the rationale for this change ? While I understand that you would be allowing more nodes to be profiled with this change (in particular, the ones that are created during IL gen), the intent of the original code may have been to avoid profiling any nodes that are created based on some other node, i.e. not created directly from bytecodes. I guess I'm wondering what the case is that you are going after with this change and whether this change only affects the case you are interested in.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supportsInduceOSR looks at doNotProfile flag on a bci to see if a node is created during ilgen/directly from bytecodes, to determine if OSR is possible at a given bci. The intention of this change is to ensure that the flag is correctly set, such that the supportsInduceOSR does not respond true to nodes that are created after ilgen through this API.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can initialize doNotProfile in TR_ByteCodeInfo's constructor to 1 such that we only need to set it to 0 when the node is created directly from bytecode.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not completely clear. I assume you are concerned about doNotProfile being set to false in cases when it ought to be set to true, judging by your comment around not wanting supportsInduceOSR to return true for nodes created after IL gen. Can you give me an example of code that creates a node after IL gen but was setting doNotProfile to false ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No one sets doNotProfile except Node APIs. Most Node APIs do not set this flag, but allow it to have default value, which can be true or false because we don't initialize it.

@vijaysun-omr vijaysun-omr Nov 6, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..."but I think the bytecode index can be arbitrary value because the bytecode info of the node is created using the default constructor"...is a somewhat worrisome comment.

I believe we did mandate that we pass in an originating bytecode node to node creation APIs in the past but it's possible some revision of that API in the last few years removed that mandatory requirement. I say this to suggest (hope ?) that there are'nt too many cases of creating nodes without using an API requiring an originating bytecode node but I agree it's probably better to deal with that in a separate PR. I would appreciate it if you would please create an issue for that and work on it asap once you have completed your current tasks because it's a dangerous state to be in to have uninitialized bytecode info on some nodes. On occasions the owning method is looked up based on the caller index that is part of the bytecode info and to have an uninitialized index to use to do the lookup could cause failures.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would appreciate @andrewcraik opinion on the above discussion as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the node creation/recreation APIs are a mess - there are a number of different versions with a number of different conventions and they do need to be cleaned up and rationalized, but I feel this is outside of the scope of this contribution discussion since @liqunl is proposing a perfectly reasonable pair of helpers as part of the OSR infrastructure.

The cases where the bytecode info won't be initialized - the cases of createWithSymRef which pass a NULL originating bytecode are especially problematic because the children are not available for the bytecode info. In general, bugs around the creation of nodes with uninitialized BCI have been seen a few times in the last year and indicate the API needs tightening up. That being said I don't think making the profiling flag handling explicit like has been done is wrong - it makes it clear what value it should have. I would argue that taking BCI info from a child isn't the most clear behavior either.

The issue with the node API is a much bigger one, but it will not be a quick fix. I think this is a good candidate for an architecture meeting and some help wanted items once we figure out which APIs are dangerous and need to be removed and what conventions we will apply to the handling of bytecodeinfo on recreate etc.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liqunl please open an issue for fixing cases when we have uninitialized bytecode info on nodes.

@0xdaryl please consider this for discussion at the one of the upcoming OMR architecture meetings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#3153 is created for further discussion on bytecode info correctness.

Comment thread compiler/il/OMRNode.cpp
TR_ASSERT(originalNode != NULL, "trying to recreate node from a NULL originalNode.");
if (originalNode->getOpCodeValue() == op)
{
if (!originalNode->hasSymbolReference() || newSymRef != originalNode->getSymbolReference())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again could you please elaborate on why you are making this change ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rational behind this change and the change above is that doNotProfile is used by supportsInduceOSR to determine if a node is created in ILGen. Nodes created in ilgen implies that the semantics of the node is equivalent of the semantics of the bytecode. If a node is transmuted, its semantics may have changed and OSR may not be possible.

Comment thread compiler/il/OMRNode.cpp Outdated
bool
OMR::Node::isLoadOfStaticFinalField()
{
if (self()->getOpCode().isLoadDirect() &&

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to check that a static field is loaded via a direct load on all platforms, in particular on System Z. I believe we have an api to load statics indirectly as well (exposing the address as the child of the indirect load on platforms where we want to common the address).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out, we do use indirect load for a static field on Z9. I will remove this check. Just for a record, the tree format of an indirect load of static field is

xloadi <static field symref>
   loadaddr <class statics>

@vijaysun-omr

Copy link
Copy Markdown
Contributor

It might be helpful for those reading this PR and not as familiar with OSR if you spelled out how the functionality implemented by the two new helper calls you are introducing was implemented prior to these changes. This could also help motivate the need for these changes.

Liqun Liu added 2 commits November 6, 2018 14:18
Add a markdown file with basic documentation that explains the
potentialOSRPointHelper and osrFearPointHelper.

Signed-off-by: Liqun Liu <liqunl@ca.ibm.com>
- Add a helper named osrFearPointHelper to mark an OSR fear point, a
  place that has been optimized with runtime assumptions and requires
  protection of OSR points. This helper helps to locate actual fear
  points, which as part of the program can be optimized, and allows the
  separation of OSR dependant optimization and OSR guard insertion.

- Add a helper named potentialOSRPointHelper to serve as a marker of OSR
  points. Currently, OSR points consist of calls, asyncchecks and
  monitor enters. As part of the program, they can be optimized and no
  longer be identified as OSR points. However, places where they reside
  may still be OSR possible. This helper works as a bookkeeping tool to
  track place that are OSR possible.

- Fix some node APIs such that doNotProfile is set only for nodes
  created during ilgen. doNotProfile on a bytecode info is looked at by
  OSR APIs to determine if a node is created in ilgen. Only nodes
  created in ilgen will have the OSR bookkeeping  and are able to
  support OSR transition. Thus, nodes created after ILGen supposed to
  have doNotProfile set and cannot be OSR points.

Signed-off-by: Liqun Liu <liqunl@ca.ibm.com>
@liqunl

liqunl commented Nov 6, 2018

Copy link
Copy Markdown
Contributor Author

@vijaysun-omr I changed the commit message to explain why the two helpers are needed. More detailed explaining is covered in the markdown file which is added by the other commit of this PR.

@vijaysun-omr

Copy link
Copy Markdown
Contributor

@genie-omr build all

@liqunl

liqunl commented Nov 7, 2018 •

Copy link
Copy Markdown
Contributor Author

@vijaysun-omr Build has passed, and I've created #3153 to address the bytecode info issue. Could you merge this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants