Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for const fn type_name #63084

Open
2 of 4 tasks
oli-obk opened this issue Jul 28, 2019 · 29 comments
Open
2 of 4 tasks

Tracking issue for const fn type_name #63084

oli-obk opened this issue Jul 28, 2019 · 29 comments
Labels
A-const-eval Area: Constant evaluation, covers all const contexts (static, const fn, ...) B-unstable Blocker: Implemented in the nightly compiler and unstable. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC Libs-Small Libs issues that are considered "small" or self-contained Libs-Tracked Libs issues that are tracked on the team's project board. requires-nightly This issue requires a nightly compiler in some way. S-tracking-needs-summary Status: It's hard to tell what's been done and what hasn't! Someone should do some investigation. T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@oli-obk
Copy link
Contributor

oli-obk commented Jul 28, 2019

This is a tracking issue for making and stabilizing type_name as const fn. It is not clear whether this is sound. Needs some T-lang discussion probably, too.

Steps needed:

@oli-obk oli-obk added E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. A-const-fn C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC labels Jul 28, 2019
@RalfJung
Copy link
Member

This function can change its output between rustc compilations

Can the output change between compiling a library crate, and a binary crate using that library? Or only when switching rustc versions?

@oli-obk
Copy link
Contributor Author

oli-obk commented Jul 28, 2019

Can the output change between compiling a library crate, and a binary crate using that library?

No, we're using global paths now.

Or only when switching rustc versions?

Yes, there needs to be a change in rustc that causes a change in output.

@Centril Centril added T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Jul 28, 2019
@TankhouseAle
Copy link
Contributor

TankhouseAle commented Jul 28, 2019

I might take a look at this. (I implemented the support for using the actual type_name intrinsic in const fn contexts, if you don't remember me, haha.)

I think we definitely know it works at compile time though, don't we? That's what this test I added at the time checks for.

@TankhouseAle
Copy link
Contributor

Sorry it took me a bit to get to this. Just opened a PR.

Centril added a commit to Centril/rust that referenced this issue Jul 30, 2019
…, r=oli-obk

`const fn`-ify `std::any::type_name` as laid out in rust-lang#63084

A test, based on the one I added when I implemented support for the underlying `core::intrinsics::type_name` being allowed in `const fn` contexts, is included.
@Centril Centril added the requires-nightly This issue requires a nightly compiler in some way. label Jul 30, 2019
Centril added a commit to Centril/rust that referenced this issue Jul 30, 2019
…, r=oli-obk

`const fn`-ify `std::any::type_name` as laid out in rust-lang#63084

A test, based on the one I added when I implemented support for the underlying `core::intrinsics::type_name` being allowed in `const fn` contexts, is included.
bors added a commit that referenced this issue Jul 30, 2019
Rollup of 7 pull requests

Successful merges:

 - #62293 (Unsupport the `await!(future)` macro)
 - #62469 (Add doc links to liballoc crate page)
 - #63095 (Turn `INCOMPLETE_FEATURES` into lint)
 - #63117 (Use global variable 'environ' to pass environments to rtpSpawn)
 - #63123 (`const fn`-ify `std::any::type_name` as laid out in #63084)
 - #63129 (Subslice patterns: Test passing static & dynamic semantics.)
 - #63147 (Updated RELEASES.md for 1.37.0)

Failed merges:

r? @ghost
@lambda-fairy
Copy link
Contributor

How does this break referential transparency? I'd like to see that elaborated upon.

@TankhouseAle
Copy link
Contributor

TankhouseAle commented Aug 4, 2019

I might be wrong, but I think the first "checkbox" can be checked since my PR was merged?

@lfairy:

Personally, I disagree, but some people seem to think that despite the fact Rust has fully reified generics, it's somehow a "dangerous" concept for your "average Joe/Jane" to be able to access the (100%-guaranteed-correct) name of a generic parameter.

Something about "Java programmers I used to work with did Some Bad Thing and I don't want Rust programmers to also do Some Bad Thing".

I don't know anything about Java though so I'm probably not overly qualified to get too much into that.

@jonas-schievink jonas-schievink added the B-unstable Blocker: Implemented in the nightly compiler and unstable. label Nov 26, 2019
@korken89
Copy link

Hi, is there a stabilization issue for this const-ification?

@jonas-schievink jonas-schievink removed the E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. label Mar 21, 2020
@oli-obk
Copy link
Contributor Author

oli-obk commented Apr 6, 2020

This is the tracking issue, so it will get closed on stabilization. I don't know what the next steps for stabilizing it are though. The discussion around referential transparency needs to be had (although with specialization it'd become obsolete, because then we'd get this feature anyway, but in a less convenient way). I think posting use cases would be very helpful for the discussion. Since this works with feature gates, you can always show a use case with nightly.

@korken89
Copy link

korken89 commented Apr 6, 2020

Hi, certainly I can write about a use case that I am working on!

I'm part of the Embedded WG where I am currently working on an instrumentation/logging tool for embedded targets. Here, as you probably know, we have strict requirements on code size as the micro controllers have very little memory.
And a huge hog of memory is when strings are placed in memory.

To this end I place formating strings and (I want) type strings in an INFO linker section so they are available in the ELF but not loaded onto the target. That is the following code as an example:

// test1 is some variable
mylog::log!("Look what I got: {}", &test1);

//
// Expands to:
//

const fn get_type_str<T>(_: &T) -> &'static str {
    // type_name needs to be a const fn for this to work
    core::any::type_name::<T>()
}

// ugly hack to tranform `&'static str` -> `[u8; str.len()]`
union Transmute<T: Copy, U: Copy> {
    from: T,
    to: U,
}

const FMT: &'static str = "Look what I got: {}";
const TYP: &'static str = get_type_str(&test1);

// Transform string into byte arrays at compile time so 
// they can be placed in a specific memory region

// Formating string placed in a specific linker section
#[link_section = ".some_info_section"]
static F: [u8; FMT.as_bytes().len()] = unsafe {
    *Transmute::<*const [u8; FMT.len()], &[u8; FMT.as_bytes().len()]> {
        from: FMT.as_ptr() as *const [u8; FMT.as_bytes().len()],
    }
    .to
};

// Type string placed in a specific linker section
#[link_section = ".some_info_section"]
static T: [u8; TYP.as_bytes().len()] = unsafe {
    *Transmute::<*const [u8; TYP.len()], &[u8; TYP.as_bytes().len()]> {
        from: TYP.as_ptr() as *const [u8; TYP.as_bytes().len()],
    }
    .to
};

// Here we send where the strings are stored in the ELF + the data to the host 
write_frame_to_host(&F, &T, &test1)

Link script:

SECTIONS {
  .some_info_section (INFO) :
  {
    *(.some_info_section .some_info_section.*);
  }
}

By doing this the strings are available in the generated ELF but not loaded onto the target, and the type can be reconstructed through DWARF with help of the type string.
And with complex types the space used for the type is getting very large for example any type that uses GenericArray (which we use extensively).

Currently the string is placed in .rodata which means that the compiler fully knows that this is a static string, and I made an issue on how to control where strings are placed which can use a trick outlined in here: #70239
However this trick only works if the string is available in const context.

This is why I am asking about stabilization here.

@That3Percent
Copy link

I would also like to get this stabilized and can present a use-case.

I'm working on firestorm: a low overhead intrusive flame graph profiler. I created firestorm because existing solutions have too much overhead to be useful for Tree-Buf: the self-describing serialization system that produces files smaller than GZip faster than uncompressed formats.

Part of keeping the overhead of firestorm low is to delay string formatting/allocation until outside of the profiled region. So, there is an enum that allows for arbitrary amounts of &'static str to be concatenated. In order to keep the amount of data writing to a minimum during the execution of the profiled region, we only write &'static EventData so that EventData can be large. Supporting concatenation of fn name and struct name for a method is only possible if type_name is a const fn.

TLDR: I need this to be const fn for performance.

@That3Percent
Copy link

To elaborate here's code I would like to have be able to compile:

#[macro_export]
macro_rules! profile_method {
    ($($t:tt)*) => {
        let _firestorm_method_guard = {
            const FIRESTORM_STRUCT_NAME: &'static str = ::std::any::type_name::<Self>();
            let event_data: &'static _ = &$crate::internal::EventData::Start(
                $crate::internal::Start::Method {
                    signature: stringify!($($t)*),
                    typ: FIRESTORM_STRUCT_NAME,
                }
            );
            $crate::internal::start(event_data);
            $crate::internal::SpanGuard
        };
    };
}

It seems there are at least 2 problems here though. First, is that type_name is not a const fn. The second is that the compiler complains about the use of Self as the generic for the const fn. I do not really understand why that's a problem when each monomorphized function could have its own value here.

@oli-obk
Copy link
Contributor Author

oli-obk commented Jun 25, 2020

@That3Percent I don't think the Self part is something that can be solved at present, even if we stabilize type_name. You can test this out by using nightly and activating the feature gate for const_type_name

@That3Percent
Copy link

@oli-obk Yes, you are right. Both issues would need to be resolved to support my use-case in the way that I would like.

@oli-obk
Copy link
Contributor Author

oli-obk commented Jul 12, 2020

I found a bug with type_name: type_name works in array lengths of arrays used in associated consts, even if it is type_name::<Self>. This doesn't even work for size_of. The bug is that type_name just returns _ in that case:

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=4404a9b3635b4bf5047c2493ac9a5082

If you comment out the first panic, you get the correct type name.

@korken89
Copy link

Ping again on the status of stabilization?

@RalfJung
Copy link
Member

type_id (#77125) has a bunch of issues which is why its stabilization actually got reverted; not sure if those issues also apply to type_name.

@Hoverbear
Copy link
Contributor

Would you believe I clicked the wrong link in Rustdoc? Sorry.

@TheNeikos
Copy link
Contributor

Is there any outlook on stabilization/progress towards it?

@oli-obk
Copy link
Contributor Author

oli-obk commented Oct 18, 2022

There's a rendering bug and a soundness bug that we need to fix before we can stabilize. Both are linked from the main post

@terrarier2111
Copy link
Contributor

There's a rendering bug and a soundness bug that we need to fix before we can stabilize. Both are linked from the main post

The soundness bug seems to have been fixed by now, and all other linked issues also seem to be fixed, is there any particular roadblock remaining?

SimonSapin added a commit to apollographql/router that referenced this issue May 29, 2024
Reproduction: `cargo test -p apollo-federation -- -q`

## Background

#5157 introduced a new kind of
snapshot test for the Rust query planner. Test schemas are specified as sets
of subgraph schemas, so composing them is needed. Since we don’t have
composition in Rust yet, the tests rely on JS composition through Rover.
To avoid a dependency on Rover on CI and for most contributors,
the composed supergraph are cached in the repository. Rover use is opt-in
and only required when a cached file is missing or out of date.
(Composition inputs are hashed.)

The file name is derived (through a macro) from the test function name.
To detect name conflicts, a static `OnceLock<Mutex<HashSet<&'static str>>>`
is used to ensure no name is used more than once.

## The problem

The static hash set relies on all snapshot tests running in the same process.
This is the case with `cargo test`, but `cargo nextest run` as used on CI
isolates every test in its own process. This breaks conflict detection
of cache file names for composed supergraph schemas, since each test only
"sees" itself in the static hash set.

#5240 introduced a name conflict:
composition is used in a function called twice with different arguments.
Because nextest was used both locally and on CI, the conflict went undectected.
As a result, running `cargo test` on dev fails because the conflict is detected.

## This PR

This PR fixes this case of cache file name conflict, but conflict detection
is still broken with nextest.

As a result it’s possible that this kind of conflict could happen again
and be merged undectected.

## Non-solutions tried

* Nextest has a notion of [test groups](https://nexte.st/book/test-groups),
  but they don’t appear to let multiple tests run in the same process

* Instead of relying on the runtime side effect of tests, could conflict
  detection rely on enumerating tests at compile time?
  The [`linkme` crate](https://crates.io/crates/linkme) is a building block
  Router used to register Rust plugins. It could be used here in all
  composition inputs can be made const, but `std::any::type_name` used
  in a macro to extract the current function name is not `const fn`
  [yet](rust-lang/rust#63084)

## Potential solutions

* Remove the cache and accept the dependency on Rover for testing.
  This impacts CI and all contributors.

* Require every `planner!` macro invocation to specify an explicit
  cache file name instead of relying on the function name.
  Then conflict detection can use `linkme`.

* Move conflict detection to a separate test that something something
  parses Rust source files of other tests something
SimonSapin added a commit to apollographql/router that referenced this issue May 29, 2024
Reproduction: `cargo test -p apollo-federation -- -q`

## Background

#5157 introduced a new kind of snapshot test for the Rust query planner. Test schemas are specified as sets of subgraph schemas, so composing them is needed. Since we don’t have composition in Rust yet, the tests rely on JS composition through Rover. To avoid a dependency on Rover on CI and for most contributors, the composed supergraph are cached in the repository. Rover use is opt-in and only required when a cached file is missing or out of date. (Composition inputs are hashed.)

The file name is derived (through a macro) from the test function name. To detect name conflicts, a static `OnceLock<Mutex<HashSet<&'static str>>>` is used to ensure no name is used more than once.

## The problem

The static hash set relies on all snapshot tests running in the same process. This is the case with `cargo test`, but `cargo nextest run` as used on CI isolates every test in its own process. This breaks conflict detection of cache file names for composed supergraph schemas, since each test only "sees" itself in the static hash set.

#5240 introduced a name conflict: composition is used in a function called twice with different arguments. Because nextest was used both locally and on CI, the conflict went undectected. As a result, running `cargo test` on dev fails because the conflict is detected.

## This PR

This PR fixes this case of cache file name conflict, but conflict detection is still broken with nextest.

As a result it’s possible that this kind of conflict could happen again and be merged undectected.

## Non-solutions tried

* Nextest has a notion of [test groups](https://nexte.st/book/test-groups), but they don’t appear to let multiple tests run in the same process

* Instead of relying on the runtime side effect of tests, could conflict detection rely on enumerating tests at compile time? The [`linkme` crate](https://crates.io/crates/linkme) is a building block Router used to register Rust plugins. It could be used here in all composition inputs can be made const, but `std::any::type_name` used in a macro to extract the current function name is not `const fn` [yet](rust-lang/rust#63084)

## Potential solutions

* Remove the cache and accept the dependency on Rover for testing. This impacts CI and all contributors.

* Require every `planner!` macro invocation to specify an explicit cache file name instead of relying on the function name. Then conflict detection can use `linkme`.

* Move conflict detection to a separate test that something something parses Rust source files of other tests something
lrlna pushed a commit to apollographql/router that referenced this issue Jun 3, 2024
Reproduction: `cargo test -p apollo-federation -- -q`

## Background

#5157 introduced a new kind of snapshot test for the Rust query planner. Test schemas are specified as sets of subgraph schemas, so composing them is needed. Since we don’t have composition in Rust yet, the tests rely on JS composition through Rover. To avoid a dependency on Rover on CI and for most contributors, the composed supergraph are cached in the repository. Rover use is opt-in and only required when a cached file is missing or out of date. (Composition inputs are hashed.)

The file name is derived (through a macro) from the test function name. To detect name conflicts, a static `OnceLock<Mutex<HashSet<&'static str>>>` is used to ensure no name is used more than once.

## The problem

The static hash set relies on all snapshot tests running in the same process. This is the case with `cargo test`, but `cargo nextest run` as used on CI isolates every test in its own process. This breaks conflict detection of cache file names for composed supergraph schemas, since each test only "sees" itself in the static hash set.

#5240 introduced a name conflict: composition is used in a function called twice with different arguments. Because nextest was used both locally and on CI, the conflict went undectected. As a result, running `cargo test` on dev fails because the conflict is detected.

## This PR

This PR fixes this case of cache file name conflict, but conflict detection is still broken with nextest.

As a result it’s possible that this kind of conflict could happen again and be merged undectected.

## Non-solutions tried

* Nextest has a notion of [test groups](https://nexte.st/book/test-groups), but they don’t appear to let multiple tests run in the same process

* Instead of relying on the runtime side effect of tests, could conflict detection rely on enumerating tests at compile time? The [`linkme` crate](https://crates.io/crates/linkme) is a building block Router used to register Rust plugins. It could be used here in all composition inputs can be made const, but `std::any::type_name` used in a macro to extract the current function name is not `const fn` [yet](rust-lang/rust#63084)

## Potential solutions

* Remove the cache and accept the dependency on Rover for testing. This impacts CI and all contributors.

* Require every `planner!` macro invocation to specify an explicit cache file name instead of relying on the function name. Then conflict detection can use `linkme`.

* Move conflict detection to a separate test that something something parses Rust source files of other tests something
Geal pushed a commit to apollographql/router that referenced this issue Jun 10, 2024
Reproduction: `cargo test -p apollo-federation -- -q`

#5157 introduced a new kind of snapshot test for the Rust query planner. Test schemas are specified as sets of subgraph schemas, so composing them is needed. Since we don’t have composition in Rust yet, the tests rely on JS composition through Rover. To avoid a dependency on Rover on CI and for most contributors, the composed supergraph are cached in the repository. Rover use is opt-in and only required when a cached file is missing or out of date. (Composition inputs are hashed.)

The file name is derived (through a macro) from the test function name. To detect name conflicts, a static `OnceLock<Mutex<HashSet<&'static str>>>` is used to ensure no name is used more than once.

The static hash set relies on all snapshot tests running in the same process. This is the case with `cargo test`, but `cargo nextest run` as used on CI isolates every test in its own process. This breaks conflict detection of cache file names for composed supergraph schemas, since each test only "sees" itself in the static hash set.

#5240 introduced a name conflict: composition is used in a function called twice with different arguments. Because nextest was used both locally and on CI, the conflict went undectected. As a result, running `cargo test` on dev fails because the conflict is detected.

This PR fixes this case of cache file name conflict, but conflict detection is still broken with nextest.

As a result it’s possible that this kind of conflict could happen again and be merged undectected.

* Nextest has a notion of [test groups](https://nexte.st/book/test-groups), but they don’t appear to let multiple tests run in the same process

* Instead of relying on the runtime side effect of tests, could conflict detection rely on enumerating tests at compile time? The [`linkme` crate](https://crates.io/crates/linkme) is a building block Router used to register Rust plugins. It could be used here in all composition inputs can be made const, but `std::any::type_name` used in a macro to extract the current function name is not `const fn` [yet](rust-lang/rust#63084)

* Remove the cache and accept the dependency on Rover for testing. This impacts CI and all contributors.

* Require every `planner!` macro invocation to specify an explicit cache file name instead of relying on the function name. Then conflict detection can use `linkme`.

* Move conflict detection to a separate test that something something parses Rust source files of other tests something
theJC pushed a commit to theJC/router that referenced this issue Jun 10, 2024
Reproduction: `cargo test -p apollo-federation -- -q`

## Background

apollographql#5157 introduced a new kind of snapshot test for the Rust query planner. Test schemas are specified as sets of subgraph schemas, so composing them is needed. Since we don’t have composition in Rust yet, the tests rely on JS composition through Rover. To avoid a dependency on Rover on CI and for most contributors, the composed supergraph are cached in the repository. Rover use is opt-in and only required when a cached file is missing or out of date. (Composition inputs are hashed.)

The file name is derived (through a macro) from the test function name. To detect name conflicts, a static `OnceLock<Mutex<HashSet<&'static str>>>` is used to ensure no name is used more than once.

## The problem

The static hash set relies on all snapshot tests running in the same process. This is the case with `cargo test`, but `cargo nextest run` as used on CI isolates every test in its own process. This breaks conflict detection of cache file names for composed supergraph schemas, since each test only "sees" itself in the static hash set.

apollographql#5240 introduced a name conflict: composition is used in a function called twice with different arguments. Because nextest was used both locally and on CI, the conflict went undectected. As a result, running `cargo test` on dev fails because the conflict is detected.

## This PR

This PR fixes this case of cache file name conflict, but conflict detection is still broken with nextest.

As a result it’s possible that this kind of conflict could happen again and be merged undectected.

## Non-solutions tried

* Nextest has a notion of [test groups](https://nexte.st/book/test-groups), but they don’t appear to let multiple tests run in the same process

* Instead of relying on the runtime side effect of tests, could conflict detection rely on enumerating tests at compile time? The [`linkme` crate](https://crates.io/crates/linkme) is a building block Router used to register Rust plugins. It could be used here in all composition inputs can be made const, but `std::any::type_name` used in a macro to extract the current function name is not `const fn` [yet](rust-lang/rust#63084)

## Potential solutions

* Remove the cache and accept the dependency on Rover for testing. This impacts CI and all contributors.

* Require every `planner!` macro invocation to specify an explicit cache file name instead of relying on the function name. Then conflict detection can use `linkme`.

* Move conflict detection to a separate test that something something parses Rust source files of other tests something
@Bromeon
Copy link

Bromeon commented Jul 9, 2024

With the recent const {} stabilization, this would prove very useful in static assertion messages.

Are there more known roadblocks to const fn type_name<T>()?

@tgross35
Copy link
Contributor

tgross35 commented Jul 9, 2024

@ickk
Copy link
Contributor

ickk commented Oct 2, 2024

What were the fruits of that discussion? What blockers, if any, were named?

Can this tracking issue be updated with any outstanding concerns

@RalfJung
Copy link
Member

RalfJung commented Oct 2, 2024

The main concern that got raised is that if we give people type_name but not type_id, they will start using type_name comparison to check if two types are equal. The docs explicitly say not to do that, but it'll happen anyway when the proper alternative (type_id comparison) is simply not available.

@oli-obk

Note that stabilization isn't happening any time soon. This function can change its output between rustc compilations and allows breaking referential transparency. Doing so at compile-time has not been discussed fully and has various points that are problematic.

Do you remember what this is about? In which sense does this function break referential transparency? We do have to ensure that the result is consistent across all crates in a crate graph but I assume that is already the case? Or are there -C/-Z flags that can change the behavior of this function?

@oli-obk
Copy link
Contributor Author

oli-obk commented Oct 2, 2024

-Zverbose used to change the output. We've since switched to -Zverbose-types, but I do not know if that still affects the output

@RalfJung
Copy link
Member

RalfJung commented Oct 2, 2024

Hm, that seems like a potential soundness concern then, doesn't it? The same const could produce different results when it is sometimes evaluated with that flag and sometimes without.

Is there any more systematic way we can avoid accidentally having const-eval depend on such flags? This is a really easy mistake to make and it'll hardly ever be detected... but it is a soundness issue.

@ickk
Copy link
Contributor

ickk commented Oct 2, 2024

@RalfJung

The main concern that got raised is that if we give people type_name but not type_id, they will start using type_name comparison to check if two types are equal. The docs explicitly say not to do that, but it'll happen anyway when the proper alternative (type_id comparison) is simply not available.

If the type_id solution is still a long way out, and if the main concern is misuse of the api: would it be possible to add a lint against that misuse to try to control for it?

@RalfJung RalfJung added A-const-eval Area: Constant evaluation, covers all const contexts (static, const fn, ...) and removed A-const-fn labels Dec 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-const-eval Area: Constant evaluation, covers all const contexts (static, const fn, ...) B-unstable Blocker: Implemented in the nightly compiler and unstable. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC Libs-Small Libs issues that are considered "small" or self-contained Libs-Tracked Libs issues that are tracked on the team's project board. requires-nightly This issue requires a nightly compiler in some way. S-tracking-needs-summary Status: It's hard to tell what's been done and what hasn't! Someone should do some investigation. T-lang Relevant to the language team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests