Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a start on the JIT IR and the trace builder. #930

Merged
merged 2 commits into from
Jan 8, 2024

Conversation

vext01
Copy link
Contributor

@vext01 vext01 commented Dec 19, 2023

The JIT IR is designed to be small. There are two kinds of instruction:

  • short instructions, with inlined operands.
  • long ones (unimplemented as of yet)

Ideally, function, block and instruction IDs (which are indices read from the on-disk AOT IR) would be either converted to references as we decode, but this would require changes to the decoder. We may have to ditch deku:

sharksforarms/deku#383

ykrt/src/compile/jitc_yk/aot_ir.rs Outdated Show resolved Hide resolved
ykrt/src/compile/jitc_yk/aot_ir.rs Outdated Show resolved Hide resolved
ykrt/src/compile/jitc_yk/aot_ir.rs Outdated Show resolved Hide resolved
ykrt/src/compile/jitc_yk/aot_ir.rs Show resolved Hide resolved
ykrt/src/compile/jitc_yk/aot_ir.rs Outdated Show resolved Hide resolved
ykrt/src/compile/jitc_yk/jit_ir.rs Show resolved Hide resolved
ykrt/src/compile/jitc_yk/jit_ir.rs Outdated Show resolved Hide resolved

impl Operand {
pub(crate) fn new(kind: OpKind, val: u64) -> Self {
if val < 2u64.pow(15) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hang on, isn't this exceeding the range of u64? What is the aim here? I wonder if what was intended was to reserve u64::MAX as a special value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's checking if the operand's payload can fit in a small operand, where we have only 15-bits of wiggle room.

(a long instruction, once implemented, would allow larger operands)

I missed the 15 in my "kill all magic numbers" pass too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(2^15 should fit in a u64 though, or I've gone mad)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, got it. I'll leave the comment here so we remember to resolve it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(2^15 should fit in a u64 though, or I've gone mad)

I misread it because I'm so unfamiliar with that way of doing bit patterns -- mea culpa!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a constant in a34b500

ykrt/src/compile/jitc_yk/trace_builder.rs Outdated Show resolved Hide resolved
ykrt/src/compile/jitc_yk/trace_builder.rs Outdated Show resolved Hide resolved
@vext01
Copy link
Contributor Author

vext01 commented Dec 20, 2023

@ltratt last two commits ok for you?

@@ -89,7 +89,8 @@ pub enum Operand {

impl Operand {
pub(crate) fn new(kind: OpKind, val: u64) -> Self {
if val < 2u64.pow(u32::try_from(SHORT_OPERAND_VALUE_SIZE).unwrap()) {
// check if the operand's value can fit in a short operand.
if val & (u64::MAX << SHORT_OPERAND_VALUE_SIZE) == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still not obvious to me what SIZE is (OK, OK, I know now, but I'll forget later, because it's unclear). Can we put all the maths in the constant at the top of the file when it will become much clearer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did eb47831 not work?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK the docs on SIZE help, but let's still stick all the maths in the constant (if nothing else, it makes clearer a) what the values actually are b) they're compile-time constants).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would need to be another PR as there are lots of computations we'd want to move to the top.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of them are introduced in this PR aren't they? Or are they preexisting?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are, yes, but if you want them in this PR then it will have to be next year now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, promise me you'll fix it then :)

@ltratt
Copy link
Contributor

ltratt commented Dec 20, 2023

Please squash.

vext01 and others added 2 commits December 20, 2023 12:09
The JIT IR is designed to be small. There are two kinds of instruction:
 - short instructions, with inlined operands.
 - long ones (unimplemented as of yet)

Ideally, function, block and instruction IDs (which are indices read
from the on-disk AOT IR) would be either converted to references as we
decode, but this would require changes to the decoder. We may have to
ditch deku:

sharksforarms/deku#383

Co-authored-by: Lukas Diekmann <[email protected]>
@vext01
Copy link
Contributor Author

vext01 commented Dec 20, 2023

splat.

@vext01
Copy link
Contributor Author

vext01 commented Dec 20, 2023

OK, promise me you'll fix it then :)

#931

@ltratt
Copy link
Contributor

ltratt commented Dec 20, 2023

Thanks!

@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 20, 2023
@ptersilie
Copy link
Contributor

Huh, we haven't touched any of the old JIT stuff so this test shouldn't fail. It may be related to the bug I'm working on at the moment. Though I failed to reproduce it locally even with try_repeat 100.

@vext01
Copy link
Contributor Author

vext01 commented Dec 21, 2023

I agree that this failure seems unrelated. Maybe kick it again and see if it merges?

@ltratt ltratt added this pull request to the merge queue Dec 21, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 21, 2023
@vext01
Copy link
Contributor Author

vext01 commented Dec 21, 2023

don't know how to handle operand: %1 = load ptr, ptr @shadowstack_0, align 8

again...

@ltratt
Copy link
Contributor

ltratt commented Dec 22, 2023

Yep, so this isn't a transient error...

@vext01
Copy link
Contributor Author

vext01 commented Dec 23, 2023

It's probably this: #909

I can look into it in January.

@vext01
Copy link
Contributor Author

vext01 commented Jan 8, 2024

I just forced a build of this and it succeeded:
https://ci.soft-dev.org/#/builders/1/builds/3288

So I think the failure is in fact transient, maybe it is #909.

It's probably gonna need some try_repeat and thinking hard to fix.

What shall we do with this PR? Retry?

@vext01
Copy link
Contributor Author

vext01 commented Jan 8, 2024

(Note that #909 predates this PR)

@ptersilie
Copy link
Contributor

Actually, I was working on a bug before Christmas that might fix this. I still have to make a test for the PR. Let's see if that works before looking into this further.

@vext01
Copy link
Contributor Author

vext01 commented Jan 8, 2024

Since this PR appears unrelated to the failure, should we retry it?

@ptersilie
Copy link
Contributor

Yeah, let's go for it.

@ltratt ltratt added this pull request to the merge queue Jan 8, 2024
Merged via the queue into ykjit:master with commit 81e1b4b Jan 8, 2024
2 checks passed
vext01 added a commit to vext01/yk_pv that referenced this pull request Jan 9, 2024
@vext01 vext01 mentioned this pull request Jan 9, 2024
nmdis1999 pushed a commit to nmdis1999/yk_fork that referenced this pull request Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants