Skip to content

Implement evaluator for inlining of StringLatin1.inflate - #12657

Merged
0xdaryl merged 1 commit into
eclipse-openj9:masterfrom
BradleyWood:strinflate
Nov 19, 2021
Merged

0xdaryl merged 1 commit into
eclipse-openj9:masterfrom
BradleyWood:strinflate

Conversation

@BradleyWood

Copy link
Copy Markdown
Member

Implement evaluator for inlining of StringLatin1.inflate

Signed-off-by: BradleyWood bradley.wood@ibm.com

@0xdaryl 0xdaryl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good. Good job figuring it all out.

I made a few comments throughout, but here are some of the bigger ones.

Please use the Reg suffix on all variables holding a TR::Register for consistency.

I would appreciate a few more descriptive comments in the main body of the implementation that describes the approach. My experience with similar sequences in other areas of the code is that you may spend a bit of time re-grokking the code when you come back to it in the future.

If a node is commoned (i.e., its reference count is > 1), when you evaluate that node into a virtual register you can't make modifications to that register without affecting the other consumers of that node. The exception to this is if this is the last reference to a node (i.e., node->getReferenceCount() == 1)

In your case, once you evaluate the source offset, the destination offset, and length children into a register, you can't modify that register if there are future consumers of that node.

What we typically do in this situation is a "clobber evaluate" rather than an "evaluate" (e.g., gprClobberEvaluate() vs evaluate()). A clobber evaluate is intended to be used in situations where the register will be clobbered by evaluator. It will check the reference count before evaluating, and if > 1 then it will evaluate and then insert a register copy and hand you back the copied register. You are free to manipulate the copy as you wish.

There are some opportunities to use smaller instructions, make better use of the addressing modes to make the code a bit cleaner, and move code to only the paths that it will be executed on. I reworked parts of the implementation as an example. Please consider this as a guide only as I may not have fully grokked the full details of the implementation.

   headerOffsetConst = sizeof contiguous array header
   vectorLengthConst = 16

   lea   destBufferReg, [destBufferReg + 1*destOffsetReg + headerOffsetConst ]
   lea   srcOffsetReg, [srcOffsetReg + headerOffsetConst]
   lea   lengthReg, [lengthReg + 1*srcOffsetReg + -vectorLengthConst]

   cmp   srcOffsetReg, lengthReg  
   jg endLoop                  ; zero length or too small for vector length
                               ; jg instead of jge so that length=16 is done in the vector loop

   pxor  zeroReg, zeroReg

startLoop:

   movdqu xmmHighReg, [srcBufferReg + 1*srcOffsetReg]
   movdqu xmmLowReg, xmmHighReg

   punpckhbw xmmLowReg, zeroReg
   movdqu [destBufferReg + vectorLengthConst], xmmLowReg

   punpcklbw xmmHighReg, zeroReg
   movdqu [destBufferReg], xmmHighReg

   add   srcOffsetReg, vectorLengthConst
   add   destBufferReg, 2*vectorLengthConst

   cmp   srcOffsetReg, lengthReg
   jle startLoop

endLoop:

   add   lengthReg, vectorLengthConst

residueLoop:
   cmp   srcOffsetReg, lengthReg
   jge done

   ; Rather than copy up to 15 bytes individually, there may be an opportunity to use
   ; some subset of an 8 byte move + 4 byte move + 2 byte move + 1 byte move
   ; to move data more efficiently.  However, the cost of determining what to do at
   ; runtime and branching around the parts that don't apply may be prohibitive.
   ; If you haven't already, you might want to think about what that instruction
   ; sequence would look like and we can reason about its performance implications.

   movzx tempReg, [srcBufferReg + 1*srcOffsetReg]
   mov   [destBufferReg], tempReg
   inc   srcOffsetReg
   add   destOffsetReg, 2
   jmp residueLoop

done:

@fjeremic fjeremic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this PR should wait until @dchopra001 has checked in the rest of the JCL changes and an updated suite of unit tests which caught some corner cases during development. @BradleyWood did you also get a chance to measure the performance of this implementation using the JMH benchmark @dchopra001 has?

@BradleyWood
BradleyWood force-pushed the strinflate branch 2 times, most recently from 3e82543 to 5db2cbb Compare May 17, 2021 13:25
@BradleyWood

Copy link
Copy Markdown
Member Author

I have changed the way the residual bytes (< 16) are copied to use the method suggested by @0xdaryl . The duffs device method did not yield good performance results. I have attached the benchmarks below. Performance is good, except for 0 length strings, which I suspect are not too common. I added a branch to compare the length of the string with zero, though it still could not match the non-intrinsic implementation.

image

Also,

   lea   destBufferReg, [destBufferReg + 1*destOffsetReg + headerOffsetConst ]
   lea   srcOffsetReg, [srcOffsetReg + headerOffsetConst]
   lea   lengthReg, [lengthReg + 1*srcOffsetReg + -vectorLengthConst]

This would require extra registers because they would need to be clobbered.

@0xdaryl

0xdaryl commented May 17, 2021

Copy link
Copy Markdown
Contributor

Thanks for the analysis. Can you paste the generated code from your residue implementation using Duff's Device so I can see the instruction sequence please?

@BradleyWood

Copy link
Copy Markdown
Member Author

I have not had the chance to clean it up yet, so bear with me.

   TR::LabelSymbol *afterDuffDevice = generateLabelSymbol(cg);
   TR::LabelSymbol *duffDevice = generateLabelSymbol(cg);
   TR::LabelSymbol *afterCopy8 = generateLabelSymbol(cg);

   generateRegRegInstruction(SUB4RegReg, node, lengthReg, srcOffsetReg, deps, cg);

   generateRegImmInstruction(CMP4RegImm4, node, lengthReg, 8, deps, cg);
   generateLabelInstruction(JL1, node, afterDuffDevice, deps, cg);

   generateRegMemInstruction(MOVQRegMem, node, xmmLowReg, generateX86MemoryReference(srcBufferReg, srcOffsetReg, 0, headerOffsetConst, cg), deps, cg);
   generateRegRegInstruction(PUNPCKLBWRegReg, node, xmmLowReg, zeroReg, deps, cg);
   generateMemRegInstruction(MOVDQUMemReg, node, generateX86MemoryReference(destBufferReg, destOffsetReg, 0, headerOffsetConst, cg), xmmLowReg, deps, cg);
   generateRegImmInstruction(SUB4RegImm4, node, lengthReg, 8, deps, cg);
   generateRegImmInstruction(ADD4RegMem, node, srcOffsetReg, 8, deps, cg);
   generateRegImmInstruction(ADD4RegMem, node, srcOffsetReg, 16, deps, cg);

   generateLabelInstruction(LABEL, node, afterCopy8, deps, cg);

   generateRegRegImmInstruction(IMUL4RegRegImm4, node, scratchReg, lengthReg, -10, cg);
   generateRegImmInstruction(ADD4RegImm4, node, scratchReg, 80, deps, cg);

   generateRegMemInstruction(LEA4RegMem, node, lengthReg,  generateX86MemoryReference(duffDevice->getLabelSymbol(), cg), deps, cg);
   generateRegRegInstruction(ADD4RegReg, node, lengthReg, scratchReg, deps, cg);
   generateRegInstruction(JMP4, node, lengthReg, deps, cg);

   generateLabelInstruction(LABEL, node, duffDevice, deps, cg);

   for (int i = 0; i < 7; i++) {
      generateRegMemInstruction(MOVZXReg2Mem1, node, xmmLowReg, generateX86MemoryReference(srcBufferReg, srcOffsetReg, 0, headerOffsetConst + 7 - i, cg), cg);
      generateMemRegInstruction(S2MemReg, node, generateX86MemoryReference(destBufferReg, destOffsetReg, 0, headerOffsetConst + 2 * (7 - i), cg), xmmLowReg, cg);
   }

   generateLabelInstruction(LABEL, node, afterDuffDevice, deps, cg);
   afterDuffDevice->setEndInternalControlFlow();

@fjeremic

Copy link
Copy Markdown
Contributor

I have attached the benchmarks below. Performance is good, except for 0 length strings, which I suspect are not too common.

@dchopra001 can you please paste the string length distribution chart you have from DayTrader7? I believe size 0 strings were fairly common even though I'd agree you would not expect that to be the case.

@genie-openj9

Copy link
Copy Markdown

Can one of the admins verify this patch?

@dchopra001

Copy link
Copy Markdown
Contributor

Here's the distribution chart for Strings length >=0 && length <= 20 (seen by Liberty in a DayTrader7 run):

Screen Shot 2021-05-17 at 10 27 45 AM

  • ~96.5% of all Strings were observed to be within this length
  • Strings of length 2 make up ~32% of all strings
  • Strings of length 1 and 2 make up about 45-46% of all strings

@fjeremic

Copy link
Copy Markdown
Contributor

Right! It was not 0 size. It was 1 & 2 that were by far the most common. My apologies I had that wrong. Given the results then we should perform better, though it would be best if we saw bigger improvements in the short string sizes.

@BradleyWood
BradleyWood force-pushed the strinflate branch 2 times, most recently from 45973c0 to 53a7d16 Compare May 27, 2021 13:48
@BradleyWood

Copy link
Copy Markdown
Member Author

Changed TR_X86OpCodes to TR::InstOpCode::Mnemonic

@BradleyWood
BradleyWood force-pushed the strinflate branch 2 times, most recently from 1834734 to 2917266 Compare June 22, 2021 15:02
@BradleyWood

Copy link
Copy Markdown
Member Author

@fjeremic

Copy link
Copy Markdown
Contributor

Jenkins test sanity xlinux jdk8,jdk11

@fjeremic

fjeremic commented Jul 5, 2021

Copy link
Copy Markdown
Contributor

Jenkins test sanity xlinux jdk8,jdk11

@0xdaryl

0xdaryl commented Oct 18, 2021

Copy link
Copy Markdown
Contributor

@BradleyWood : can you summarize the next steps for this PR? Is it ready for final review?

@BradleyWood

Copy link
Copy Markdown
Member Author

@BradleyWood : can you summarize the next steps for this PR? Is it ready for final review?

I think so, but there are a few things I would like to discuss before you proceed.

@BradleyWood

Copy link
Copy Markdown
Member Author

@0xdaryl Should be ready for review

@fjeremic

Copy link
Copy Markdown
Contributor

Jenkins test sanity xlinux jdk8,jdk11

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think !TR::Compiler->om.canGenerateArraylets() should also be fatal asserted here to make it clear up front that discontiguous arrays are not supported.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call this scratchReg for consistency.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L11937: this should be a TR_ASSERT_FATAL_WITH_NODE so there is some failure context.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you write this as:

const int copy_instruction_size =  5  // sizeof(MOVZXReg2Mem1)
                                  +4; // sizeof(S2MemReg)

so it is clear how the length constant is derived?

Delete the single line comment on L12040.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call this doneLabel

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call this copyResidueLabel

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call this afterCopy8Label

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work without a relocation for AOT compiles (because the "duffDevice" address is only valid for this method, not if it is AOT loaded later).

Have a look at how the jump table in the tableEvaluator is implemented:

https://github.com/eclipse/omr/blob/d3053f922dd8f81535ea3396fbab3e8a51ac6e83/compiler/x/codegen/ControlFlowEvaluator.cpp#L468

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure? The code generated is rip-relative,

48 8d 05 0e 00 00 00 // lea rax,[rip+0xe]

Unless this is not guaranteed...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say "guaranteed", but I'd say it is extremely likely to produce RIP-relative code. I'll double check the mem ref code.

Note that RIP-relative addressing is not available on 32-bit, so if this is something that will run on JDK8 then you'll need another solution (like the table evaluator)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This StringLatin1.inflate does not exist of JDK8

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a fatal assert at the beginning of this function then guarding against a 32-bit invocation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signed-off-by: BradleyWood <bradley.wood@ibm.com>
@0xdaryl

0xdaryl commented Nov 8, 2021

Copy link
Copy Markdown
Contributor

Jenkins test sanity xlinux,win,osx jdk8,jdk17

@0xdaryl

0xdaryl commented Nov 9, 2021

Copy link
Copy Markdown
Contributor

@fjeremic : any chance you could approve your requested changes?

@fjeremic fjeremic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gald to see this going in!

Comment on lines +12040 to +12047
const int copy_instruction_size = 5 // size of MOVZXReg2Mem1
+4; // size of S2MemReg

// since copy_instruction_size could change depending on which registers are allocated to scratchReg, srcOffsetReg and destOffsetReg
// we reserve them to be eax, ecx, edx, respectively

generateRegRegImmInstruction(TR::InstOpCode::IMUL4RegRegImm4, node, lengthReg, lengthReg, -copy_instruction_size, cg);
generateRegImmInstruction(TR::InstOpCode::ADD4RegImm4, node, lengthReg, copy_instruction_size * 7, cg);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if there is a better way to do this. Seems fragile to changes but at least we have the comment explaining why it is so.

@BradleyWood BradleyWood Nov 12, 2021 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, there really is no way to know the size of the instructions without reserving the registers because the REX prefix might be used. Maybe later we could modify the register dependencies to request any register that does not require the rex prefix. At least that might alleviate some register pressure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants