Skip to content

[rustc_public] Enhance PassMode API#159359

Open
celinval wants to merge 3 commits into
rust-lang:mainfrom
celinval:feat/cast-passmode
Open

[rustc_public] Enhance PassMode API#159359
celinval wants to merge 3 commits into
rust-lang:mainfrom
celinval:feat/cast-passmode

Conversation

@celinval

Copy link
Copy Markdown
Contributor

This change is split into two different commits:

  1. Expand PassMode::Cast with CastTarget: This is basically a 1:1 mapping from the internal compiler representation.
  2. Replace Opaque with ArgAttributes in PassMode: Expose the non-LLVM specific ArgAttributes information. Leave the internal representation private for now in case we do want to expand its implementation later.

celinval added 2 commits July 15, 2026 23:52
Replace the opaque representation of `PassMode::Cast` with a
structured `CastTarget` type that exposes the register layout
used by the platform ABI. Add `Uniform`, `Reg`, and `RegKind`
types so tools can inspect how arguments are mapped to registers.

Add `CastTarget::size()` and `Uniform::reg_count()` helpers.
Add a test covering cast on args, returns, mixed register kinds,
multiple arguments, and register exhaustion causing stack spill.
Expose argument ABI attributes through a structured type instead of
opaque debug strings. ArgAttributes provides accessors for the
extension mode (zero/sign-extend) and pointee alignment, which are
needed by tools doing their own code generation.

This removes all uses of Opaque from the abi module.
@rustbot

rustbot commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

This PR changes rustc_public

cc @oli-obk, @ouz-a, @makai410

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 16, 2026
@rustbot

rustbot commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

r? @makai410

rustbot has assigned @makai410.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: project-rustc-public
  • project-rustc-public expanded to celinval, makai410, oli-obk, scottmcm
  • Random selection from makai410, oli-obk, scottmcm

@workingjubilee workingjubilee Jul 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file has incorrect statements such as

    /// The `abi` defines how this data is passed between functions.
    pub abi: ValueAbi,

See #132246 for more on why this is wrong.

I would honestly rather you not "enhance" anything here and set almost this entire file on fire here because it's just going to make me actually rewriting this harder. We could make more accurate information available in the future, but it's unclear why anyone would benefit from the current information because they lack the full context to interpret it.

View changes since the review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the context. Having to reverse engineer the compiler is not an easy task, so please do point out any inaccuracies. Most of the comments here I wrote based on the comments from rustc_target and from analyzing the generated FnAbi for different function signatures.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dearly sympathize because my task of figuring out what the hell is going on involves reverse-engineering LLVM. 🙃

At some point I just sort of started assuming everything I am reading is probably wrong, unfortunately.

@workingjubilee workingjubilee Jul 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My question here is mostly: What question is someone trying to answer that they think this information would be of use to them? If they're trying to do a basic ABI equality check then smuggling the data around opaquely with mostly no transformation and exposing a PartialEq impl that interrogates the innards seems more useful.

@workingjubilee workingjubilee Jul 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To elaborate: The rustc_target and rustc_abi crates were mostly written ages ago by people who are not easy to ask questions of anymore, in a single-backend context, where x86-64 was our only tier 1 target. It was then incrementally edited, including by people such as myself, in ways that were "good enough", and our unclear understandings documented as truth because no one was around anymore who completely knew exactly what was going on and we could usefully discuss things with. "cg_ssa" (rustc_codegen_ssa) was introduced as an incomplete abstraction layer over LLVMIR, and to the extent it has abstracted anything, it has made everything in rustc_abi and rustc_target wronger.

That is why asking questions about things below FnAbi is probably not a good idea, unless it's about a fundamental like Size or Align. Obviously those have to be correct. And we've... tried to make LayoutData... vaguely intelligible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use case is tools that build their own backends or analyzers on top of MIR and need to generate compatible call sequences or how the code interacts via FFI without depending on LLVM. They need to know: how many arguments there are (including implicit ones), whether a value is passed by copy or by pointer, and what type transformations the ABI requires. FnAbi/PassMode captures those decisions.

I've pushed a commit fixing the inaccuracies you pointed out — renamed ValueAbi to ValueRepr and LayoutShape::abi to LayoutShape::value_repr to align with the BackendRepr rename from #132246, and corrected the doc comments that were claiming layout fields define calling behavior. Happy to address any other inaccuracies you spot.

Let me know if there are changes to how we expose the data that would make it easier to evolve the internals.

@workingjubilee workingjubilee Jul 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, if anyone wants to build a codegen backend, they should be focusing on using cg_ssa. That doesn't allow multi-version stability, but there is a reason I am saying it. If it is insufficient or problematic, it should be improved until it serves the need.

Analysis alone, however, is indeed the intended use-case of things here.

In either case, if they want a precise answer that doesn't depend on LLVM that can reason about things like "generating correct call sequences", they're Kinda Screwed Right Now. They would need to also implement the exact logic, essentially, that LLVM does, in order to get a consistently correct answer, because of how we kinda only model "ABI modulo LLVM".

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. It might be sufficient if they want only a purely abstract matching of Rust's function ABI to a C signature? That might be wrong but it should be correct-enough.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Rust ABI there are things that can't be represented as a valid C signature. For example there are cases where we use more return registers than the platform C ABI allows, but LLVM supports representing it anyway.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context. To summarize where I think we are: this PR adds information that is currently missing from rustc_public — the structure of PassMode::Cast, argument attributes, and corrected documentation. The data itself is not incorrect; it accurately reflects the compiler's calling convention decisions.

Users may still need to do further processing depending on their use case (as cg_ssa does today), and the Rust ABI can produce things that don't map cleanly to a C signature. The data shouldn't be taken at face value as "this is exactly what happens on the wire" — but it is the correct input for tools that need to reason about argument layout and pass modes.

@workingjubilee

Copy link
Copy Markdown
Member

What is the purpose of the information here? It honestly is mostly not information that can be used correctly by people who aren't willing to just look directly at the LLVMIR and emitted assembly, which will actually tell them what is going on.

@workingjubilee

workingjubilee commented Jul 16, 2026

Copy link
Copy Markdown
Member

Like, all of the annotations that we emit do not have a precise relationship with what LLVM does, because LLVM... interprets... various IR patterns differently, often in a way that depends on which machine backend you are using, specifics of the exact target in question, and even the position of the argument (well before you run out of registers, mind!). That is, to put it simply: rustc_abi::PassMode::Direct plus rustc_abi::BackendRepr::Scalar does not guarantee it is passed directly. Thus why I say that anyone interested in the answers here should just dump the LLVM text/assembly. Then you can see both what we asked for and what the actual results were.

@rustbot

This comment has been minimized.

@celinval celinval force-pushed the feat/cast-passmode branch from 59b3ad1 to ae8aeeb Compare July 16, 2026 07:14
BREAKING CHANGE: `ValueAbi` is renamed to `ValueRepr` and the
`LayoutShape::abi` field is renamed to `LayoutShape::value_repr`.

The old name was misleading: this type does not describe how values
are passed in function calls (that is `PassMode`), it is a hint for
how backends should represent values (as scalars, vectors, or
aggregates). This aligns with the internal rename from `Abi` to
`BackendRepr`.

Also fixes several doc comments that incorrectly claimed layout
fields define calling behavior.
@celinval celinval force-pushed the feat/cast-passmode branch from ae8aeeb to f340101 Compare July 16, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants