Skip to content

RISC-V: reimplement c_cInterpreter - #12982

Merged
gacholio merged 5 commits into
eclipse-openj9:masterfrom
janvrany:pr/riscv-reimplement-cinterpreter
Nov 6, 2021
Merged

gacholio merged 5 commits into
eclipse-openj9:masterfrom
janvrany:pr/riscv-reimplement-cinterpreter

Conversation

@janvrany

Copy link
Copy Markdown
Contributor

This PR reimplements c_cInterpreter in assembly in order to support transition from JIT code to interpreter and back.

@janvrany

Copy link
Copy Markdown
Contributor Author

jenkins test sanity xlinux jdk11

@janvrany janvrany mentioned this pull request Jun 16, 2021
@janvrany

Copy link
Copy Markdown
Contributor Author

Looks like jenkins is not listening to me...

@pshipton

Copy link
Copy Markdown
Member

Looks like jenkins is not listening to me...

Only committers can trigger PR testing.

@pshipton

pshipton commented Jun 16, 2021 •

Copy link
Copy Markdown
Member

Since this change is specific to RISC-V, I don't see why we need to test sanity on xlinux. I assume this is tested on RISC-V?

@pshipton

Copy link
Copy Markdown
Member

@gacholio are you able to review the assembler?

@janvrany

Copy link
Copy Markdown
Contributor Author

@pshipton : it does change common CMake. In previous PR @0xdaryl did that just to make sure it does not break the build.
I "tested" the change on RISC-V, but just by running some Java code, I did not run sanity nor any other test suites.

@gacholio

Copy link
Copy Markdown
Contributor

@gacholio are you able to review the assembler?

I'll try and take a look next week - I'm not familiar with the instruction set.

@janvrany

Copy link
Copy Markdown
Contributor Author

@janvrany

Copy link
Copy Markdown
Contributor Author

Polite ping.

Comment thread runtime/vm/riscvcinterp.m4 Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems confusing - why is 2 being stored in slot 18?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, I think it may be incorrect - the save area referred to by these macros is for the stack frame described by:

#elif defined(J9VM_ARCH_RISCV) /* J9VM_ARCH_ARM */
	UDATA preservedGPRs[11]; /* x2, x8, x9, and x18-x25  */
	U_8 preservedFPRs[32 * 8]; /* f0-f31 */
	UDATA jitGPRs[32]; /* x0-x31 */
	U_8 jitFPRs[32 * 8]; /* f0-f31 */
	U_8 padding[8]; /* padding to 16-byte boundary */

So there are only 11 slots - index 18 is somewhere in the middle of the FPR save area.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right, this seems wrong. Will fix it soon. Thanks!

@janvrany
janvrany force-pushed the pr/riscv-reimplement-cinterpreter branch from f434361 to 55abf6e Compare July 28, 2021 19:24
@janvrany

Copy link
Copy Markdown
Contributor Author

@gacholio I just pushed new version (55abf6e). Changes:

  1. Add two more slots to preservedGPRs in j9nonbuilder.h to make space for s10 and s11. I did not realize this is for saving C regs (s10 and s11 are used as thread pointer / javaSP in Java's "private" linkage"
  2. Reduce number of slots in preservedFPRs to 12 since in C ABI only fs0 - fs11 are preserved.
  3. Fix c_cInterpreter to use correct slots and also to save FPRs.

Hope this is better.

@gacholio

Copy link
Copy Markdown
Contributor

I am going on vacation for a week. I will try to take a look, but no guarantees.

@janvrany

Copy link
Copy Markdown
Contributor Author

No problem at all, take it easy. I have bug(s) in JIT to chase. Thanks!

The JIT register save area has (must have) space for all 32 registers
so we shall save them in slots corresponding with their architectural
numbers (s0 is x8, therefor should be save to slot 8)
On RISC-V there's no designated FP register (and when used, it's s0).
Signed-off-by: Jan Vrany <jan.vrany@fit.cvut.cz>
@janvrany
janvrany force-pushed the pr/riscv-reimplement-cinterpreter branch from 55abf6e to 329249e Compare August 4, 2021 12:02
@janvrany

janvrany commented Aug 4, 2021

Copy link
Copy Markdown
Contributor Author

Just pushed an update to fix few more issues I found when debugging JIT (1a2f97d, 5263cb2). Hope I got it right this time.

@gacholio

gacholio commented Sep 2, 2021

Copy link
Copy Markdown
Contributor

This looks good in general. There are lots of seemingly extra blank lines. It would also be good to describe the register layout - is ra a register in itself, or is it an alias for one of the numbered registers? It's stored in JIT GPR save slot 1, but preserved in the interp prologue in GPR save slot 12.

I also don't understand the JIT register layout - there's a large gap in the saves between s1 and s2. Really, s0 should be in save slot 0, etc in order for the stack walker to work with the JIT register maps.

@janvrany

Copy link
Copy Markdown
Contributor Author

@gacholio Sorry for the late reply,

I'm not pretending I understand all the consequence of the layout - all my understanding of how this works come from reading the (some) of the code. Here I merely did what AArhc64 does.

Below is the table summarizing registers and their roles:


Arch name ABI name Syst. linkage role Preserved? Priv. linkage rol (+) cframe_preservedFPRs index cframe_jitGPRs index
x0 zero Zero --
x1 ra Return address No 12 1
x2 sp Stack pointer Yes 2
x3 gp Global pointer -- 3
x4 tp Thread pointer -- 4
x5 t0 Temporary register No 5
x6 t1 Temporary register No 6
x7 t2 Temporary register No 7
x8 s0 Callee-saved register Yes 0 8
x9 s1 Callee-saved register Yes 1 9
x10 a0 Arg & ret register No 10
x11 a1 Arg & ret register No 11
x12 a2 Argument register No 12
x13 a3 Argument register No 13
x14 a4 Argument register No 14
x15 a5 Argument register No 15
x16 a6 Argument register No 16
x17 a7 Argument register No 17
x18 s2 Callee-saved register Yes 2 18
x19 s3 Callee-saved register Yes 3 19
x20 s4 Callee-saved register Yes 4 20
x21 s5 Callee-saved register Yes 5 21
x22 s6 Callee-saved register Yes 6 22
x23 s7 Callee-saved register Yes 7 23
x24 s8 Callee-saved register Yes 8 24
x25 s9 Callee-saved register Yes 9 25
x26 s10 Callee-saved register Yes vmThread 10 26
x27 s11 Callee-saved register Yes javaSP 11 27
x28 t3 Temporary register No 28
x29 t4 Temporary register No 29
x30 t5 Temporary register No 30
x31 t6 Temporary register No 31

(+) empty means same role as in system linkage


Basically:

  • for cframe_preservedFPRs there's only 13 slots, for s0 - s11, s0 being saved in slot 0, s1 in slot 1 and so on and ra in slot 12,.
  • for cframe_ jitGPRs there's full 32 slots for all registes and each register is stored at index corresponding with architectural register number, that is, s0 is x8 therefore get saved into slot 8.
    The gap is there because in there's a gap between s1 and s2. This is because of compressed instruction set in order to allow compressed instruction to access at least some of "s" registers and some of "t" registers. We do not support compressed instructions yet, but I can imagine this will be on the table in order improve code density (in some distant future)
    I choose to to use matching indexes as it seemed to be logical and easy to work with. Now if there are other contraints such as stack walker, then we can change it but I was not aware of this and would have to first learn how these work : - )

@gacholio

Copy link
Copy Markdown
Contributor

Ah I see the confusion between the two register names - I suggest simply working in terms of x registers where possible (possibly using the ABI names for doing C calls). You can define your own aliases in the m4 code if need be.

@janvrany

janvrany commented Sep 20, 2021 •

Copy link
Copy Markdown
Contributor Author

I suggest simply working in terms of x registers where possible (possibly using the ABI names for doing C calls).

I'd rather work in terms of ABI names - I use ABI names consistently among the codebase. Compiler code uses them too. gas supports both. GDB disassembly uses ABI names. (Yeah, now I see that some comments refer to x-names, I'd rather fix those).
Why do you think using arch. name (x) would be better?

You can define your own aliases in the m4 code if need be.

Sorry, I do not understand why would one need aliases. gas undestands ABI names just fine.

@janvrany

Copy link
Copy Markdown
Contributor Author

What can be done is to change GPR_SAVE_OFFSET macro so one would use same offset as for JIT_GPR_SAVE_OFFSET, that is, s0 which is x8 would go to GPR_SAVE_OFFSET(8). Would that make it clearer / less confusing?
(need to check whether this is possible in m4).

@gacholio

Copy link
Copy Markdown
Contributor

The JIT does not follow any ABI but its own, so I don't personally think it makes sense to use the ABI names there, as they provide no benefit.

I would use the x names for anything dealing with JIT registers (it's important the the register numbers match up with the appropriate bits in the register map, so numbering them simply makes sense).

Using the ABI names makes sense when dealing with the C ABI, like the preserved registers in the C callin, the arg registers and return address.

Is the JIT code using the same return address register as C?

Perhaps someone from the JIT team should also comment. @0xdaryl @fjeremic

@janvrany

janvrany commented Sep 20, 2021 •

Copy link
Copy Markdown
Contributor Author

The private linkage (JIT linkage) as I implemented it uses the same registers for same purpose as the system linkage,
except s10 and s11 - see

https://github.com/janvrany/openj9/blob/jv/riscv-devel/runtime/compiler/riscv/codegen/RVPrivateLinkage.cpp#L64-L100

and

https://github.com/eclipse/omr/blob/master/compiler/riscv/codegen/RVSystemLinkage.cpp#L68-L104

I think AArch64 does similar thing.

makes sense to use the ABI names there, as they provide no benefit.
I would argue that the benefit is that

  1. ABI names are simpler to understand (at the very least for me). I think it is easier to remember that a stands for argument and s foe saved register than to remember that x10 is argument but x9 is saved.
  2. The compiler code uses ABI names exclusively, that is both in JIT and in disassembly. GDB uses ABI names in disassembly so it is easier to match compiler code / compiler tracelog with disassembly one would see in GDB.

it's important the the register numbers match up with the appropriate bits in the register map, so numbering them simply makes sense)

Sorry, I still do not understand what this exactly means. Do you mean saved / volatile registers must be numbered consecutively? That is, we cannot have the "gap" in JIT-saved registers? I think need to do my homework first and understand that part of the code.

Is the JIT code using the same return address register as C?

Yes. Also same argument / return GPRs / FPRs, same temporary registers, same saved registers (except s10 and s11 which are marked as reserved for the JIT is JIT-generated code does not touch them at all),

@gacholio

Copy link
Copy Markdown
Contributor

Bit 0 in the register map maps to x0, bit 1 to x1, etc, so at the very least, you need to be able to derive the true register number from the ABI alias when constructing the map (I would be surprised if the internal representation of a register didn't know it's real index).

The register map is for GC purposes - it has a bit for every register that can possibly hold an object pointer.

The preserved map is unrelated, but still ordered (e.g. 101 would indicate that s0 and s2 are preserved, with no empty slot for s1 between).

So, when preserving registers in the helpers, to me it would be much clearer to store x0 into JIT GPR save slot 0, etc, rather than using the ABI names there, as there would end up being a confusing mix of the names.

If this really bothers you, perhaps add extra helper macros to store the ABI registers by number that add the appropriate offset into the numbered GPR area - this would be a lot clearer than store s0 into GPR slot 8, etc.

@janvrany

Copy link
Copy Markdown
Contributor Author

able to derive the true register number from the ABI alias
I believe that's easy.

(I would be surprised if the internal representation of a register didn't know it's real index).

Indeed it does. Register definitions are generated from https://github.com/janvrany/omr/blob/jv/riscv-devel/compiler/riscv/codegen/riscv-regs.h

would be much clearer to store x0 into JIT GPR save slot 0

This is exacty what the code does at is is now except is uses ABI names, not x names.

perhaps add extra helper macros to store the ABI registers by number that add the appropriate offset into the numbered GPR area

Do you mean that for - say SAVE_C_VOLATILE_REGS instead of (current):
https://github.com/janvrany/openj9/blob/jv/riscv-devel/runtime/oti/riscvhelpers.m4#L94-L113
one would write:

sd  ra, JIT_GPR_SAVE_SLOT_ra

    sd  t0,  JIT_GPR_SAVE_SLOT_t1
    sd  t1,  JIT_GPR_SAVE_SLOT_21
    ...
    sd  t6,  JIT_GPR_SAVE_SLOT_t6

    sd  a0,  JIT_GPR_SAVE_SLOT_a0
    sd  a1,  JIT_GPR_SAVE_SLOT_a1
    ...
    sd  a7,  JIT_GPR_SAVE_SLOT_a7

Would this be fine with you? This is certainly fine with me.

@gacholio

Copy link
Copy Markdown
Contributor

I was thinking something different, but that seems just as good.

@janvrany

Copy link
Copy Markdown
Contributor Author

I was thinking something different, but that seems just as good.

Perfect, will do soon! Thanks a lot!

@fjeremic

Copy link
Copy Markdown
Contributor

Is the JIT code using the same return address register as C?

Perhaps someone from the JIT team should also comment. @0xdaryl @fjeremic

In general no. For private (JIT to JIT) linkage we use the same registers on the same processor irrespective of OS, however the system linkage may use different registers depending on OS. A clear example of this can be found on the system linkage where on Linux the return register is GPR2 and on z/OS (64-bit) the return register is GPR3. In JIT private linkage the return register is GPR2 always:

setIntegerReturnRegister (TR::RealRegister::GPR2 );
setLongLowReturnRegister (TR::RealRegister::GPR3 );
setLongHighReturnRegister(TR::RealRegister::GPR2 );
setLongReturnRegister (TR::RealRegister::GPR2 );

There is a reference document for system linkage here:
https://github.com/eclipse/omr/blob/master/doc/diagnostics/CallingConventions.md

…ic register name

This commit changes both macros so that they take symbolic ABI register names
(`s0`, `a5`) rather than index to save area.

The conversion from register name to offset is entirely hidden to users
of these macros.
@janvrany

Copy link
Copy Markdown
Contributor Author

@gacholio It took longer than expected. In the end I settled passing symbolic register names to GPR_SAVE_SLOT() and convert the name to index into save are there. For example:

sd ra, GPR_SAVE_SLOT(ra)
sd s0, GPR_SAVE_SLOT(s0)
sd s1, GPR_SAVE_SLOT(s1)
sd s2, GPR_SAVE_SLOT(s2)
sd s3, GPR_SAVE_SLOT(s3)
sd s4, GPR_SAVE_SLOT(s4)

@janvrany

Copy link
Copy Markdown
Contributor Author

Very polite ping.

@gacholio

Copy link
Copy Markdown
Contributor

Sorry for the (long) delay.

@gacholio gacholio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One missing thing - the module.xml must be updated. Please search for rv64cinterp and replace with riscvcinterp. In the same file, there's a reference to the nojit version that should be deleted.

@janvrany

janvrany commented Nov 5, 2021

Copy link
Copy Markdown
Contributor Author

@gacholio : Thanks! Fixed in bfaadef. Hope I did the right thing.

@gacholio

gacholio commented Nov 5, 2021

Copy link
Copy Markdown
Contributor

jenkins compile zlinux jdk8

@gacholio
gacholio merged commit 301530f into eclipse-openj9:master Nov 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants