const-generic array splitting

[jplatte]

Creating a new issue about const-generic array splitting methods since it's been discussed on the array_chunks PR, where it really doesn't belong.

We probably want

impl [T] {
    fn split_array<const N: usize>(&self) -> Option<(&[T; N], &[T])> { ... }
}

// requires more const generics work to be accepted by rustc AFAIK
impl<T, const N: usize> [T; N] {
    fn split_array<const M: usize>(&self) -> (&[T; M], &[T; {N - M}]) { ... }
}

The possibility of a variadic array splitting function was also mentioned (at least that's my interpretation) but since there's not even an active RFC about variadic generics, I think that's a bit out of scope. The only thing one could do once the second split_array above becomes possible is add split_array_2, split_array_3 and so on.

[leonardo-m]

We also need two or more different use cases.

[withoutboats]

@DutchGhost pointed out that this could be implemented today by taking 2 const params, but the check that they subdivide the array evenly would have to be a runtime assertion.

impl [T; N]
    fn split<N2, N3>(&self) -> (&[T; N2] &[T; N3]) {
        assert_eq!(N, N2 + N3);
        // ...
    }
}

We could add the API on nightly with an awareness that we intend to change the API someday.

[DutchGhost]

I made an implementation that errors at compiletime when you specify wrong array lengths!

use core::mem::size_of;

trait SplitCheck<const N2: usize, const N3: usize> {
    type LHS;
    type RHS;

    const ASSERT: bool;
    const __ASSERT: () = [()][(!Self::ASSERT) as usize];
}

impl<T, const N: usize, const N2: usize, const N3: usize> SplitCheck<{ N2 }, { N3 }> for [T; N] {
    type LHS = [T; N2];
    type RHS = [T; N3];

    const ASSERT: bool = size_of::<Self::LHS>() + size_of::<Self::RHS>() == size_of::<Self>();
}

impl<T, const N: usize> [T; N] {
    fn split_array<const N2: usize, const N3: usize>(&self) -> (&[T; N2], &[T; N3]) {
        let _: () = <Self as SplitCheck<{ N2 }, { N3 }>>::__ASSERT;

        let (lhs, rhs) = self.split_at(N2);

        unsafe {
            let lhs_array = &(*lhs.as_ptr().cast());
            let rhs_array = &(*rhs.as_ptr().cast());

            (lhs_array, rhs_array)
        }
    }
}

Here a playground link for a PoC: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=52d0664c958d9bca65935b8df6086c33

[jethrogb]

use case: Parsing binary data into structs that contain fixed-length arrays

[withoutboats]

I think something that uses traits to impose relational const constraints makes more sense outside of std right now.

[DutchGhost]

I think something that uses traits to impose relational const constraints makes more sense outside of std right now.

yeah, the error message you get when the assert hits is far from pretty and usefull. I'd say its a tradeoff between wanting some ugly compile error whenever the lenghts are wrong, or taking that assertion and finding out the lenghts are wrong during runtime.

One thing to make the error a little bit nicer, is to write the following:

    const __ASSERT: &'static str =
        ["The total length of the splitted arrays does not equal the length of the original!"]
            [(!Self::ASSERT) as usize];

Now, if somehow the values of N, N2 and N3 could be displayed in that message, it would be even nicer

[leonardo-m]

Compile-time errors are better. This means we need to add something to Rust to perform similar static casts in a nicer and shorter way.

[withoutboats]

@leonardo-m Aren't you the same leonardo from the internals forum who created a thread about this exact feature a week ago? I responded there with an explanation of the state of things in regard to that feature.

In any event the "correct" API is not to have two parameters and constrain them, but to derive the length of the second array by subtracting the a single parameter from the length of the main array. (-> (&[T; M], &[T; N - M])). But the current implementation of const generics does not support that.

Probably it would be better to hash out what APIs we could implement now, and whether its worth adding them to nightly, in a zulip thread than a GitHub issue.

[leonardo-m]

Aren't you the same leonardo from the internals forum who created a thread about this exact feature a week ago?

There I asked for a "where" clause on const integers, while here I've suggested about a more general const_assert that's usable in more/different cases.

[mbartlett21]

Here is an example implementation:

(EDIT: Updated features)

#![feature(const_refs_to_cell)]
#![feature(const_ptr_read)]
#![feature(const_ptr_offset)]
#![feature(generic_const_exprs)]

use core::mem::ManuallyDrop;

const fn split_arr<T, const N: usize, const M: usize>(arr: [T; N]) -> ([T; M], [T; N - M])
where
    [T; N - M]: ,
{
    let arr = ManuallyDrop::new(arr);
    let start_ptr = &arr as *const _ as *const T;
    let arr_1_ptr = start_ptr as *const [T; M];
    let arr_2_ptr = unsafe { start_ptr.add(M) } as *const [T; N - M];

    unsafe { (arr_1_ptr.read(), arr_2_ptr.read()) }
}

const ARR: [u8; 10] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

const FIRST_3: [u8; 3] = split_arr(ARR).0;

// This last one doesn't work, since rustc can't infer the constant
// const LAST_7: [u8; 7] = split_arr(ARR).1;

// We just need to specify the generic params...
const LAST_7: [u8; 7] = split_arr::<u8, 10, 3>(ARR).1;

(playground)

[jethrogb]

Is it possible to add [T; N]::split_array now but adjust the signature so that its use is always a compile-time error? I'd like to land the slice version ASAP but I don't want to run into the same problem we had with into_iter.

[jethrogb]

Another use case: converting binary data to integers using {integer}::from_Xe_bytes

[jethrogb]

Is it possible to add [T; N]::split_array now but adjust the signature so that its use is always a compile-time error? I'd like to land the slice version ASAP but I don't want to run into the same problem we had with into_iter.

This seems easy: just keep the array version unstable?

[slightlyoutofphase]

This is 100% doable in current nightly rust, in a way that simply results in an immediate compile-time error if an invalid split index is provided, and is also completely const-compatible. Here's a playground link that does it as a const impl of the following trait (which is just a stand-in for the actual [T; N] type, of course):

pub trait ArrayHelper<T, const N: usize> {
    fn split_array<const M: usize>(self) -> ([T; M], [T; N - M]);
    fn split_array_ref<const M: usize>(&self) -> (&[T; M], &[T; N - M]);
    fn split_array_mut<const M: usize>(&mut self) -> (&mut [T; M], &mut [T; N - M]);
}

[leonardo-m]

To split an array or slice with this functions I'm using:

let (a1, rest) = data.split_array_ref::<N>();
let (a2, empty) = rest.split_array_ref::<M>();
assert!(empty.is_empty());
(a1, a2)

I guess there isn't a desire for a array method similar to:

fn split_into_two_arrays<T, const N: usize, const M: usize>(data: &[T; N]) -> (&[T; M], &[T; N - M]) {

[jethrogb]

I think I'd use split_array_ref followed by try_from on the rest.

[jethrogb]

Check the tracking issue #90091 for more details on exact-length splitting for arrays.