Skip to content

Commit

Permalink
Add Bytes to_buffer and to_alloc_vec (#1231)
Browse files Browse the repository at this point in the history
### What
Add Bytes to_buffer and to_alloc_vec functions that provide access to
the bytes in local guest memory. The to_buffer function provides access
to the bytes in a statically specified buffer. The to_alloc_vec function
is available only when the alloc feature is enabled and provides access
to an alloc::vec::Vec containing the bytes.

### Why
@tomerweller was talking to me about allocator usage in dev and testing
when experimenting and costs are not that high, and it got me thinking
that I mostly don't use the allocator because it's not trivial to get
data into an allocated type. It's not that hard, but I don't think it's
obvious and it's something we need to explain to people how to do.

These functions provide two ways to get access to the data inside a
variable length Bytes value, whether using the allocator or not.

The first, `to_buffer` accepts as a constant generic parameter a buffer
size, and returns the buffer filled with the contents of the bytes along
with a `Range` that describes the slice of that buffer that contains the
bytes. The bytes may not fill the full buffer and the range that case
describes where the bytes begins (always 0) and ends. The function is
usable regardless of whether the allocator is in use, and is suitable
when a reasonable maximum size is known, but an exact size is not.

The `to_alloc_vec` function requires the `alloc` feature as it
dynamically allocates the memory required to store the bytes on the
guest side.

### Costs

Examples of costs of using these functions:

#### `to_alloc_vec`

The `to_alloc_vec` function with a small bytes value results in:

- Contract Size: 3,835 bytes
- CPU Used: 5,001,490
- Bytes Read: 3,992
- Fee Charged: 59,245

```
#[contract]
pub struct Contract;

#[contractimpl]
impl Contract {
    pub fn empty(env: Env) -> u32 {
        let bytes = Bytes::from_slice(&env, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
        let len = bytes.to_alloc_vec().as_slice().len();
        len as u32
    }
}
```

#### `to_buffer::<10>`

The `to_buffer` function with a buffer size of 10 and a small bytes
value results in:

- Contract Size: 599 bytes
- CPU Used: 3,828,048
- Bytes Read: 756
- Fee Charged: 44,351

```
#[contract]
pub struct Contract;

#[contractimpl]
impl Contract {
    pub fn empty(env: Env) -> u32 {
        let bytes = Bytes::from_slice(&env, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
        let len = bytes.to_buffer::<10>().as_slice().len();
        len as u32
    }
}
```

#### `to_buffer::<1024>`

The `to_buffer` function with a buffer size of 1024 and a small bytes
value results in:

- Contract Size: 807 bytes
- CPU Used: 3,914,448
- Bytes Read: 964
- Fee Charged: 45,418

```
#[contract]
pub struct Contract;

#[contractimpl]
impl Contract {
    pub fn empty(env: Env) -> u32 {
        let bytes = Bytes::from_slice(&env, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);
        let len = bytes.to_buffer::<10>().as_slice().len();
        len as u32
    }
}
```
  • Loading branch information
leighmcculloch authored Feb 23, 2024
1 parent 02cd291 commit 9d802fc
Show file tree
Hide file tree
Showing 4 changed files with 107 additions and 0 deletions.
67 changes: 67 additions & 0 deletions soroban-sdk/src/bytes.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
#[cfg(feature = "alloc")]
extern crate alloc;

use core::{
borrow::Borrow,
cmp::Ordering,
Expand Down Expand Up @@ -602,6 +605,70 @@ impl Bytes {
pub fn iter(&self) -> BytesIter {
self.clone().into_iter()
}

/// Copy the bytes into a buffer of given size.
///
/// Returns the buffer and a range of where the bytes live in the given
/// buffer.
///
/// Suitable when the size of the bytes isn't a fixed size but it is known
/// to be under a certain size, or failure due to overflow is acceptable.
///
/// ### Panics
///
/// If the size of the bytes is larger than the size of the buffer. To avoid
/// this, first slice the bytes into a smaller size then convert to a
/// buffer.
#[must_use]
pub fn to_buffer<const B: usize>(&self) -> BytesBuffer<B> {
let mut buffer = [0u8; B];
let len = self.len() as usize;
{
let slice = &mut buffer[0..len];
self.copy_into_slice(slice);
}
BytesBuffer { buffer, len }
}

/// Copy the bytes into a Rust alloc Vec of size matching the bytes.
///
/// Returns the Vec. Allocates using the built-in allocator.
///
/// Suitable when the size of the bytes isn't a fixed size and the allocator
/// functionality of the sdk is enabled.
#[cfg(feature = "alloc")]
#[must_use]
pub fn to_alloc_vec(&self) -> alloc::vec::Vec<u8> {
let len = self.len() as usize;
let mut vec = alloc::vec::from_elem(0u8, len);
self.copy_into_slice(&mut vec);
vec
}
}

/// A `BytesBuffer` stores a variable number of bytes, up to a fixed limit `B`.
///
/// The bytes are stored in a fixed-size non-heap-allocated structure. It is a
/// minimal wrapper around a fixed-size `[u8;B]` byte array and a length field
/// indicating the amount of the byte array containing meaningful data.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BytesBuffer<const B: usize> {
buffer: [u8; B],
len: usize,
}

impl<const B: usize> Borrow<[u8]> for BytesBuffer<B> {
/// Returns a borrow slice of the bytes stored in the BytesBuffer.
fn borrow(&self) -> &[u8] {
self.as_slice()
}
}

impl<const B: usize> BytesBuffer<B> {
/// Returns a borrow slice of the bytes stored in the BytesBuffer.
pub fn as_slice(&self) -> &[u8] {
&self.buffer[..self.len]
}
}

impl IntoIterator for Bytes {
Expand Down
2 changes: 2 additions & 0 deletions soroban-sdk/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
mod address;
mod auth;
mod budget;
mod bytes_alloc_vec;
mod bytes_buffer;
mod contract_add_i32;
mod contract_assert;
mod contract_docs;
Expand Down
15 changes: 15 additions & 0 deletions soroban-sdk/src/tests/bytes_alloc_vec.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#![cfg(feature = "alloc")]

use crate::{Bytes, Env};

#[test]
fn test_bytes_alloc_vec() {
let env = Env::default();

let bytes = Bytes::from_slice(&env, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);

assert_eq!(
&[1u8, 2, 3, 4, 5, 6, 7, 8, 9, 10],
bytes.to_alloc_vec().as_slice()
);
}
23 changes: 23 additions & 0 deletions soroban-sdk/src/tests/bytes_buffer.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
use crate::{Bytes, Env};

#[test]
fn test_bytes_buffer() {
let env = Env::default();

let bytes = Bytes::from_slice(&env, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);

assert_eq!(
&[1u8, 2, 3, 4, 5, 6, 7, 8, 9, 10],
bytes.to_buffer::<1024>().as_slice(),
);
}

#[test]
#[should_panic(expected = "range end index 10 out of range for slice of length 9")]
fn test_bytes_buffer_panic() {
let env = Env::default();

let bytes = Bytes::from_slice(&env, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]);

let _ = bytes.to_buffer::<9>();
}

0 comments on commit 9d802fc

Please sign in to comment.