Signed int keys migrate example

MIGRATING.md

@@ -6,6 +6,87 @@ This guide lists API changes between *cw-plus* major releases.
 
 ### Breaking Issues / PRs
 
+- Incorrect I32Key Index Ordering [\#489](https://github.com/CosmWasm/cw-plus/issues/489) /
+  Signed int keys order [\#582](https://github.com/CosmWasm/cw-plus/pull/582)
+
+As part of range iterators revamping, we fixed the order of signed integer keys. You shouldn't change anything in your
+code base for this, but if you were using signed keys and relying on their ordering, that has now changed for the better.
+Take into account also that **the internal representation of signed integer keys has changed**. So, if you
+have data stored under signed integer keys you would need to **migrate it**, or recreate it under the new representation.
+
+As part of this, a couple helpers for handling int keys serialization and deserialization were introduced:
+- `from_cw_bytes` Integer (signed and unsigned) values deserialization.
+- `to_cw_bytes` - Integer (signed and unsigned) values serialization.
+
+You shouldn't need these, except when manually handling raw integer keys serialization / deserialization.
+
+Migration code example:
+```rust
+#[cfg_attr(not(feature = "library"), entry_point)]
+pub fn migrate(deps: DepsMut, _env: Env, _msg: Empty) -> Result<Response, ContractError> {
+  let version: Version = CONTRACT_VERSION.parse()?;
+  let storage_version: Version = get_contract_version(deps.storage)?.version.parse()?;
+
+  if storage_version < version {
+    set_contract_version(deps.storage, CONTRACT_NAME, CONTRACT_VERSION)?;
+
+    // Do the migration
+    // Original map
+    let signed_int_map: Map<IntKeyOld<i8>, String> = Map::new("signed_int_map");
+
+    // New map (using a different namespace for safety. It could be the same with enough care)
+    let signed_int_map_new: Map<i8, String> = Map::new("signed_int_map-v2");
+
+    // Obtain all current keys (this will need to be paginated if there are many entries,
+    // i.e. i32 or i64 instead of i8).
+    // This may be gas intensive
+    let current = signed_int_map
+            .range(deps.storage, None, None, Order::Ascending)
+            .collect::<StdResult<Vec<_>>>()?;
+
+    // Store length for quality control (adjust if paginated)
+    let current_count = current.len();
+
+    // Remove the old map keys
+    for (k, _) in current.iter() {
+      signed_int_map.remove(deps.storage, (*k).into());
+    }
+
+    // Save in new format
+    for (k, v) in current.into_iter() {
+      signed_int_map_new.save(deps.storage, k, &v)?;
+    }
+
+    // Confirm old map is empty
+    if signed_int_map
+            .keys_raw(deps.storage, None, None, Order::Ascending)
+            .next()
+            .is_some()
+    {
+      return Err(StdError::generic_err("Original still not empty!").into());
+    }
+
+    // Obtain new keys, and confirm their amount.
+    // May be gas intensive.
+    let new_count = signed_int_map_new
+            .keys_raw(deps.storage, None, None, Order::Ascending)
+            .count();
+
+    if current_count != new_count {
+      return Err(StdError::generic_err(format!(
+        "Current ({}) and new ({}) counts differ!",
+        current_count, new_count
+      ))
+              .into());
+    }
+  }
+
+  Ok(Response::new())
+}
+```
+
+---
+
 - Rename cw0 to utils [\#471](https://github.com/CosmWasm/cw-plus/issues/471) / Cw0 rename [\#508](https://github.com/CosmWasm/cw-plus/pull/508)
 
 The `cw0` package was renamed to `cw-utils`. The required changes are straightforward:
@@ -193,52 +274,6 @@ index 022a4504..c7a3bb9d 100644
 
 ---
 
-- Incorrect I32Key Index Ordering [\#489](https://github.com/CosmWasm/cw-plus/issues/489) /
-Signed int keys order [\#582](https://github.com/CosmWasm/cw-plus/pull/582)
-
-As part of range iterators revamping, we fixed the order of signed integer keys. You shouldn't change anything in your
-code base for this, but if you were using signed keys and relying on their ordering, that has now changed for the better.
-Take into account also that **the internal representation of signed integer keys has changed**. So, if you
-have data stored under signed integer keys you would need to **migrate it**, or recreate it under the new representation.
-
-As part of this, a couple helpers for handling int keys serialization and deserialization were introduced:
- - `from_cw_bytes` Integer (signed and unsigned) values deserialization.
- - `to_cw_bytes` - Integer (signed and unsigned) values serialization.
-
-You shouldn't need these, except when manually handling raw integer keys serialization / deserialization.
-
-Migration code example:
-```rust
-#[cfg_attr(not(feature = "library"), entry_point)]
-pub fn migrate(deps: DepsMut, _env: Env, _msg: MigrateMsg) -> Result<Response, ContractError> {
-    let version = get_contract_version(deps.storage)?;
-    if version.contract != CONTRACT_NAME {
-        return Err(ContractError::CannotMigrate {
-            previous_contract: version.contract,
-        });
-    }
-    // Original map
-    signed_int_map: Map<i8, String> = Map::new("signed_int_map");
-    // New map
-    signed_int_map_new: Map<i8, String> = Map::new("signed_int_map-v2");
-
-    signed_int_map
-        .range_raw(deps.storage, None, None, Order::Ascending)
-        .map(|(k, v)| {
-            let signed = i8::from_be_bytes(k);
-            signed_int_map_new.save(deps.storage, signed, v);
-        })
-        .collect()?;
-
-    // Code to remove the old map keys
-    ...
-
-    Ok(Response::default())
-}
-```
-
----
-
 - `cw3-fixed-multisig` requires threshold during instantiation instead of `required_weight` parameter
 
 `Threshold` type was moved to `packages/utils` along with surrounding implementations like `ThresholdResponse` etc.

packages/storage-plus/src/de_old.rs

@@ -0,0 +1,106 @@
+use std::array::TryFromSliceError;
+use std::convert::TryInto;
+
+use cosmwasm_std::{StdError, StdResult};
+
+use crate::de::KeyDeserialize;
+use crate::keys_old::IntKeyOld;
+
+macro_rules! intkey_old_de {
+    (for $($t:ty),+) => {
+        $(impl KeyDeserialize for IntKeyOld<$t> {
+            type Output = $t;
+
+            #[inline(always)]
+            fn from_vec(value: Vec<u8>) -> StdResult<Self::Output> {
+                Ok(<$t>::from_be_bytes(value.as_slice().try_into()
+                    .map_err(|err: TryFromSliceError| StdError::generic_err(err.to_string()))?))
+            }
+        })*
+    }
+}
+
+intkey_old_de!(for i8, u8, i16, u16, i32, u32, i64, u64, i128, u128);
+
+#[cfg(test)]
+mod test {
+    use super::*;
+    use crate::keys_old::IntKeyOld;
+
+    #[test]
+    fn deserialize_integer_old_works() {
+        assert_eq!(<IntKeyOld<u8>>::from_slice(&[1]).unwrap(), 1u8);
+        assert_eq!(<IntKeyOld<i8>>::from_slice(&[127]).unwrap(), 127i8);
+        assert_eq!(<IntKeyOld<i8>>::from_slice(&[128]).unwrap(), -128i8);
+
+        assert_eq!(<IntKeyOld<u16>>::from_slice(&[1, 0]).unwrap(), 256u16);
+        assert_eq!(<IntKeyOld<i16>>::from_slice(&[128, 0]).unwrap(), -32768i16);
+        assert_eq!(<IntKeyOld<i16>>::from_slice(&[127, 255]).unwrap(), 32767i16);
+
+        assert_eq!(
+            <IntKeyOld<u32>>::from_slice(&[1, 0, 0, 0]).unwrap(),
+            16777216u32
+        );
+        assert_eq!(
+            <IntKeyOld<i32>>::from_slice(&[128, 0, 0, 0]).unwrap(),
+            -2147483648i32
+        );
+        assert_eq!(
+            <IntKeyOld<i32>>::from_slice(&[127, 255, 255, 255]).unwrap(),
+            2147483647i32
+        );
+
+        assert_eq!(
+            <IntKeyOld<u64>>::from_slice(&[1, 0, 0, 0, 0, 0, 0, 0]).unwrap(),
+            72057594037927936u64
+        );
+        assert_eq!(
+            <IntKeyOld<i64>>::from_slice(&[128, 0, 0, 0, 0, 0, 0, 0]).unwrap(),
+            -9223372036854775808i64
+        );
+        assert_eq!(
+            <IntKeyOld<i64>>::from_slice(&[127, 255, 255, 255, 255, 255, 255, 255]).unwrap(),
+            9223372036854775807i64
+        );
+
+        assert_eq!(
+            <IntKeyOld<u128>>::from_slice(&[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
+                .unwrap(),
+            1329227995784915872903807060280344576u128
+        );
+        assert_eq!(
+            <IntKeyOld<i128>>::from_slice(&[128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
+                .unwrap(),
+            -170141183460469231731687303715884105728i128
+        );
+        assert_eq!(
+            <IntKeyOld<i128>>::from_slice(&[
+                127, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255
+            ])
+            .unwrap(),
+            170141183460469231731687303715884105727i128
+        );
+        assert_eq!(
+            <IntKeyOld<i128>>::from_slice(&[
+                255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255
+            ])
+            .unwrap(),
+            -1i128,
+        );
+    }
+
+    #[test]
+    fn deserialize_broken_integer_old_errs() {
+        // One byte less fails
+        assert!(matches!(
+            <IntKeyOld<u16>>::from_slice(&[1]).err(),
+            Some(StdError::GenericErr { .. })
+        ));
+
+        // More bytes fails too
+        assert!(matches!(
+            <IntKeyOld<u8>>::from_slice(&[1, 2]).err(),
+            Some(StdError::GenericErr { .. })
+        ));
+    }
+}

packages/storage-plus/src/keys_old.rs

@@ -0,0 +1,88 @@
+use crate::de::KeyDeserialize;
+use crate::keys::Key;
+use crate::{Endian, Prefixer, PrimaryKey};
+use std::marker::PhantomData;
+
+#[derive(Clone, Debug, PartialEq, Eq)]
+pub struct IntKeyOld<T: Endian> {
+    pub wrapped: Vec<u8>,
+    pub data: PhantomData<T>,
+}
+
+impl<T: Endian> IntKeyOld<T> {
+    pub fn new(val: T) -> Self {
+        IntKeyOld {
+            wrapped: val.to_be_bytes().into(),
+            data: PhantomData,
+        }
+    }
+}
+
+impl<T: Endian> From<T> for IntKeyOld<T> {
+    fn from(val: T) -> Self {
+        IntKeyOld::new(val)
+    }
+}
+
+impl<T: Endian> From<Vec<u8>> for IntKeyOld<T> {
+    fn from(wrap: Vec<u8>) -> Self {
+        IntKeyOld {
+            wrapped: wrap,
+            data: PhantomData,
+        }
+    }
+}
+
+impl<T: Endian> From<IntKeyOld<T>> for Vec<u8> {
+    fn from(k: IntKeyOld<T>) -> Vec<u8> {
+        k.wrapped
+    }
+}
+
+// this auto-implements PrimaryKey for all the IntKeyOld types
+impl<'a, T: Endian + Clone> PrimaryKey<'a> for IntKeyOld<T>
+where
+    IntKeyOld<T>: KeyDeserialize,
+{
+    type Prefix = ();
+    type SubPrefix = ();
+    type Suffix = Self;
+    type SuperSuffix = Self;
+
+    fn key(&self) -> Vec<Key> {
+        self.wrapped.key()
+    }
+}
+
+// this auto-implements Prefixer for all the IntKey types
+impl<'a, T: Endian> Prefixer<'a> for IntKeyOld<T> {
+    fn prefix(&self) -> Vec<Key> {
+        self.wrapped.prefix()
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+
+    #[test]
+    fn u64key_old_works() {
+        let k: IntKeyOld<u64> = 134u64.into();
+        let path = k.key();
+        assert_eq!(1, path.len());
+        assert_eq!(134u64.to_be_bytes(), path[0].as_ref());
+    }
+
+    #[test]
+    fn i32key_old_works() {
+        let k: IntKeyOld<i32> = 4242i32.into();
+        let path = k.key();
+        assert_eq!(1, path.len());
+        assert_eq!(4242i32.to_be_bytes(), path[0].as_ref());
+
+        let k: IntKeyOld<i32> = IntKeyOld::<i32>::from(-4242i32);
+        let path = k.key();
+        assert_eq!(1, path.len());
+        assert_eq!((-4242i32).to_be_bytes(), path[0].as_ref());
+    }
+}

packages/storage-plus/src/lib.rs

@@ -1,4 +1,5 @@
 mod de;
+mod de_old;
 mod endian;
 mod helpers;
 mod indexed_map;
@@ -8,6 +9,7 @@ mod int_key;
 mod item;
 mod iter_helpers;
 mod keys;
+mod keys_old;
 mod map;
 mod path;
 mod prefix;
@@ -32,6 +34,7 @@ pub use keys::{I128Key, I16Key, I32Key, I64Key, I8Key};
 pub use int_key::CwIntKey;
 #[allow(deprecated)]
 pub use keys::{Prefixer, PrimaryKey, U128Key, U16Key, U32Key, U64Key, U8Key};
+pub use keys_old::IntKeyOld;
 pub use map::Map;
 pub use path::Path;
 #[cfg(feature = "iterator")]