Skip to content
This repository has been archived by the owner on Aug 18, 2020. It is now read-only.

[CBR-305] SafeCopy instances for InDb types #3381

Merged
merged 3 commits into from
Aug 23, 2018
Merged

[CBR-305] SafeCopy instances for InDb types #3381

merged 3 commits into from
Aug 23, 2018

Conversation

k0001
Copy link
Contributor

@k0001 k0001 commented Aug 10, 2018

Some things to keep in mind:

  • Round-trip QuickCheck tests are missing.

  • The basic approach to defining instances for types wrapped in InDb is described in the code:

    -- Notice that when serializing and deserializing, a type like InDb foo will
    -- distribute InDb among the non-primitive types that make up foo. This
    -- choice doesn't show up in the types.

  • There are a lot of redundant type signatures. This is deliberate, so that changing the definition of a datatype in the future triggers a type-checker error if possible.

bs :: B.ByteString <- SC.safeGet
pure (InDb (Core.RedeemPublicKey (Ed25519.PublicKey bs)))
putCopy (InDb (Core.RedeemPublicKey pk)) = SC.contain $ do
SC.safePut (Ed25519.unPublicKey pk :: B.ByteString)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


src/Cardano/Wallet/Kernel/DB/InDb.hs:264:17: warning: [-Wdeprecations]
    In the use of ‘unPublicKey’ (imported from Crypto.Sign.Ed25519):
    Deprecated: "This accessor is deprecated, and will be removed in a future version. Use @'openPublicKey'@ instead."
    |
264 |     SC.safePut (Ed25519.unPublicKey pk :: B.ByteString)
    |                 ^^^^^^^^^^^^^^^^^^^

<no location info>: error:
Failing due to -Werror.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't worry, this is WIP :)

@edsko edsko added the wip label Aug 13, 2018
@parsonsmatt parsonsmatt removed the wip label Aug 22, 2018
@parsonsmatt parsonsmatt changed the title [WIP] [CBR-305] SafeCopy instances for InDb types [CBR-305] SafeCopy instances for InDb types Aug 22, 2018
@parsonsmatt
Copy link
Contributor

QuickCheck properties are still missing, but the instances are done.

@parsonsmatt
Copy link
Contributor

Failures:

  test/MarshallingSpec.hs:259:21: 
  1) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb Address
       Falsifiable (after 1 test):
         InDb {_fromDb = Address {addrRoot = AbstractHash 0b7b8b8f843142c39c5c2e048c9acc4234ba666ac2b23590f0ad8937, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 828a22d8a78f2d6b3ef7b8d519007a0a2e3b02a560950c877963207d,CoinPortion {getCoinPortion = 968125601564503}),(AbstractHash debc0a8870d5a702d253f484e8dcb12d4f877ce0cbf55556d4e7e02a,CoinPortion {getCoinPortion = 31874398435497})])} }, addrType = ATUnknown 250}}
         Left "Failed reading: safecopy: Word8: Cannot find getter associated with this version number: Version {unVersion = 33554432}\nEmpty call stack\n" /= Right (InDb {_fromDb = Address {addrRoot = AbstractHash 0b7b8b8f843142c39c5c2e048c9acc4234ba666ac2b23590f0ad8937, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 828a22d8a78f2d6b3ef7b8d519007a0a2e3b02a560950c877963207d,CoinPortion {getCoinPortion = 968125601564503}),(AbstractHash debc0a8870d5a702d253f484e8dcb12d4f877ce0cbf55556d4e7e02a,CoinPortion {getCoinPortion = 31874398435497})])} }, addrType = ATUnknown 250}})

  test/MarshallingSpec.hs:259:21: 
  2) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb AddrAttributes
       Falsifiable (after 1 test):
         InDb {_fromDb = AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 946e1fe1a219445305db7d5f297b2e53357d2786f5eba7aa95208014,CoinPortion {getCoinPortion = 8069393574399}),(AbstractHash d1cb6f80ae98f1c74bb760d855e2c1500c7a523221dca25bc79cb18a,CoinPortion {getCoinPortion = 991930606425601})])}}
         Left "Failed reading: safecopy: Word8: Cannot find getter associated with this version number: Version {unVersion = 33554432}\nEmpty call stack\n" /= Right (InDb {_fromDb = AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 946e1fe1a219445305db7d5f297b2e53357d2786f5eba7aa95208014,CoinPortion {getCoinPortion = 8069393574399}),(AbstractHash d1cb6f80ae98f1c74bb760d855e2c1500c7a523221dca25bc79cb18a,CoinPortion {getCoinPortion = 991930606425601})])}})

  test/MarshallingSpec.hs:259:21: 
  3) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb TxAux
       Falsifiable (after 1 test):
         InDb {_fromDb = TxAux {taTx = UnsafeTx {_txInputs = TxInUnknown 45 "" :| [], _txOutputs = TxOut {txOutAddress = Address {addrRoot = AbstractHash 0683626a7b1e08c6b4eae488c2a9e33ac1b6efc10cac851ad80affc9, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 4b244488d93e308ae4681894efb10509f0dcd76833ab1cb9493816e5,CoinPortion {getCoinPortion = 979706013260319}),(AbstractHash e306cf742d32ab98d2e0a4c7eeef7f803762db0dcf28b80c2d0fed95,CoinPortion {getCoinPortion = 20293986739681})])} }, addrType = ATUnknown 50}, txOutValue = Coin {getCoin = 35575148029769243}} :| [], _txAttributes = Attributes { data: () }}, taWitness = []}}
         Left "Failed reading: safecopy: Word8: Cannot find getter associated with this version number: Version {unVersion = 33554432}\nEmpty call stack\n" /= Right (InDb {_fromDb = TxAux {taTx = UnsafeTx {_txInputs = TxInUnknown 45 "" :| [], _txOutputs = TxOut {txOutAddress = Address {addrRoot = AbstractHash 0683626a7b1e08c6b4eae488c2a9e33ac1b6efc10cac851ad80affc9, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 4b244488d93e308ae4681894efb10509f0dcd76833ab1cb9493816e5,CoinPortion {getCoinPortion = 979706013260319}),(AbstractHash e306cf742d32ab98d2e0a4c7eeef7f803762db0dcf28b80c2d0fed95,CoinPortion {getCoinPortion = 20293986739681})])} }, addrType = ATUnknown 50}, txOutValue = Coin {getCoin = 35575148029769243}} :| [], _txAttributes = Attributes { data: () }}, taWitness = []}})

  test/MarshallingSpec.hs:259:21: 
  4) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb Tx
       Falsifiable (after 1 test):
         InDb {_fromDb = UnsafeTx {_txInputs = TxInUnknown 143 "" :| [], _txOutputs = TxOut {txOutAddress = Address {addrRoot = AbstractHash bf2116e773509a1a72237106ba889a2ce2c85fab31f7daa44be25714, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Nothing, aaStakeDistribution = BootstrapEraDistr} }, addrType = ATRedeem}, txOutValue = Coin {getCoin = 883942464159857}} :| [], _txAttributes = Attributes { data: () }}}
         Left "Failed reading: safecopy: Word8: Cannot find getter associated with this version number: Version {unVersion = 33554432}\nEmpty call stack\n" /= Right (InDb {_fromDb = UnsafeTx {_txInputs = TxInUnknown 143 "" :| [], _txOutputs = TxOut {txOutAddress = Address {addrRoot = AbstractHash bf2116e773509a1a72237106ba889a2ce2c85fab31f7daa44be25714, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Nothing, aaStakeDistribution = BootstrapEraDistr} }, addrType = ATRedeem}, txOutValue = Coin {getCoin = 883942464159857}} :| [], _txAttributes = Attributes { data: () }}})

  test/MarshallingSpec.hs:259:21: 
  5) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb TxOut
       Falsifiable (after 1 test):
         InDb {_fromDb = TxOut {txOutAddress = Address {addrRoot = AbstractHash a203fb47f94429dedf459772b70868206fb77f9e2727231abd5ea6bc, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 6a8bfac5c7eae5bdc678a6f032a177d5eb4cb25f6a94eb0a2c704918,CoinPortion {getCoinPortion = 150023028157586}),(AbstractHash 7d6487666ac09e71b616e4012e83855ed4c1de54bf682936f48b5b0e,CoinPortion {getCoinPortion = 849976971842414})])} }, addrType = ATUnknown 111}, txOutValue = Coin {getCoin = 23730594662986461}}}
         Left "Failed reading: safecopy: Word8: Cannot find getter associated with this version number: Version {unVersion = 33554432}\nEmpty call stack\n" /= Right (InDb {_fromDb = TxOut {txOutAddress = Address {addrRoot = AbstractHash a203fb47f94429dedf459772b70868206fb77f9e2727231abd5ea6bc, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 6a8bfac5c7eae5bdc678a6f032a177d5eb4cb25f6a94eb0a2c704918,CoinPortion {getCoinPortion = 150023028157586}),(AbstractHash 7d6487666ac09e71b616e4012e83855ed4c1de54bf682936f48b5b0e,CoinPortion {getCoinPortion = 849976971842414})])} }, addrType = ATUnknown 111}, txOutValue = Coin {getCoin = 23730594662986461}}})

  test/MarshallingSpec.hs:259:21: 
  6) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb TxOutAux
       Falsifiable (after 1 test):
         InDb {_fromDb = TxOutAux {toaOut = TxOut {txOutAddress = Address {addrRoot = AbstractHash a203fb47f94429dedf459772b70868206fb77f9e2727231abd5ea6bc, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 6a8bfac5c7eae5bdc678a6f032a177d5eb4cb25f6a94eb0a2c704918,CoinPortion {getCoinPortion = 150023028157586}),(AbstractHash 7d6487666ac09e71b616e4012e83855ed4c1de54bf682936f48b5b0e,CoinPortion {getCoinPortion = 849976971842414})])} }, addrType = ATUnknown 111}, txOutValue = Coin {getCoin = 23730594662986461}}}}
         Left "Failed reading: safecopy: Word8: Cannot find getter associated with this version number: Version {unVersion = 33554432}\nEmpty call stack\n" /= Right (InDb {_fromDb = TxOutAux {toaOut = TxOut {txOutAddress = Address {addrRoot = AbstractHash a203fb47f94429dedf459772b70868206fb77f9e2727231abd5ea6bc, addrAttributes = Attributes { data: AddrAttributes {aaPkDerivationPath = Just (HDAddressPayload {getHDAddressPayload = ""}), aaStakeDistribution = UnsafeMultiKeyDistr (fromList [(AbstractHash 6a8bfac5c7eae5bdc678a6f032a177d5eb4cb25f6a94eb0a2c704918,CoinPortion {getCoinPortion = 150023028157586}),(AbstractHash 7d6487666ac09e71b616e4012e83855ed4c1de54bf682936f48b5b0e,CoinPortion {getCoinPortion = 849976971842414})])} }, addrType = ATUnknown 111}, txOutValue = Coin {getCoin = 23730594662986461}}}})

  test/MarshallingSpec.hs:259:21: 
  7) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb (Vector TxInWitness)
       Falsifiable (after 5 tests):
         InDb {_fromDb = [ScriptWitness (Script {scrVersion = 1, scrScript = "\STX\NUL\EOT\STX"}) (Script {scrVersion = 1, scrScript = "\NUL"}),UnknownWitnessType 220 "\SOH\STX\ETX\SOH",UnknownWitnessType 155 "\SOH\EOT\STX\STX",PkWitness (PublicKey (XPub {xpubPublicKey = "XaD\131\223\167}\248\193[\228\a\135\244K_Y\155`\155\235S\r\128\185Q\194\190\DEL\129\STX\232", xpubChaincode = ChainCode "g,\171\DEL\SIyT\130h\152\194\138\141\221Q\141\181\203\235q\160\182\179\150S\174\&0M6\184\137\183"})) (Signature (XSignature {unXSignature = "\167\162\238\235\re\ETX\211\130IM\208#\159\252\194\t\RS\EM\148H%\217f~\186\213F\153\156\188h\223\164t\nA\CAN\245V\233\&5\214v\GS\SO\185\DC1X\227/\223\225\229\167E\b\255\GS;O\183\210\n"}))]}
         Left "Failed reading: safecopy: Crypto.Sign.Ed25519.PublicKey: Cannot find getter associated with this version number: Version {unVersion = -603979776}\nFrom:\tPos.Crypto.Signing.Types.Redeem.RedeemPublicKey:\n\n" /= Right (InDb {_fromDb = [ScriptWitness (Script {scrVersion = 1, scrScript = "\STX\NUL\EOT\STX"}) (Script {scrVersion = 1, scrScript = "\NUL"}),UnknownWitnessType 220 "\SOH\STX\ETX\SOH",UnknownWitnessType 155 "\SOH\EOT\STX\STX",PkWitness (PublicKey (XPub {xpubPublicKey = "XaD\131\223\167}\248\193[\228\a\135\244K_Y\155`\155\235S\r\128\185Q\194\190\DEL\129\STX\232", xpubChaincode = ChainCode "g,\171\DEL\SIyT\130h\152\194\138\141\221Q\141\181\203\235q\160\182\179\150S\174\&0M6\184\137\183"})) (Signature (XSignature {unXSignature = "\167\162\238\235\re\ETX\211\130IM\208#\159\252\194\t\RS\EM\148H%\217f~\186\213F\153\156\188h\223\164t\nA\CAN\245V\233\&5\214v\GS\SO\185\DC1X\227/\223\225\229\167E\b\255\GS;O\183\210\n"}))]})

  test/MarshallingSpec.hs:259:21: 
  8) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb TxInWitness
       Falsifiable (after 1 test):
         InDb {_fromDb = UnknownWitnessType 181 ""}
         Left "Failed reading: safecopy: Crypto.Sign.Ed25519.PublicKey: Cannot find getter associated with this version number: Version {unVersion = -1258291200}\nFrom:\tPos.Crypto.Signing.Types.Redeem.RedeemPublicKey:\n\n" /= Right (InDb {_fromDb = UnknownWitnessType 181 ""})

  test/MarshallingSpec.hs:259:21: 
  9) Marshalling & Unmarshalling, Roundtrips, Safe Copy Roundtrip for: InDb TxIn
       Falsifiable (after 2 tests):
         InDb {_fromDb = TxInUtxo (AbstractHash 47e523ec8f2d1ae373b3db511f43603a4c7cc5f8469eff79f909ba7ac674faec) 0}
         Left "Failed reading: getCopy@AbstractHash Blake2b_256 Tx: DeserialiseFailure 8 \"AbstractHash.decode: invalid digest\"\nEmpty call stack\n" /= Right (InDb {_fromDb = TxInUtxo (AbstractHash 47e523ec8f2d1ae373b3db511f43603a4c7cc5f8469eff79f909ba7ac674faec) 0})

Neat, we got test failures 😄


instance SC.SafeCopy (InDb Core.Coin) where
getCopy = SC.contain $ do
w :: Word64 <- SC.safeGet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a 7 space indent?


instance SC.SafeCopy (InDb Txp.TxAux) where
getCopy = SC.contain $ do
InDb (tx :: Txp.Tx) <- SC.safeGet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also personally prefer 2 space indent, but this code ....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll get the style right before merging it all in. I'm mostly concerned about the correctness issues and failing tests atm

@@ -1,4 +1,5 @@
{-# OPTIONS_GHC -fno-warn-orphans #-}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this need to alllow orphans? I don't immediately see it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! We can safely remove that warning :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Copy link
Contributor

@edsko edsko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Let me take a quick look to see if the rebase would be hard.

putCopy (InDb (AddressMeta isUsed isChange)) = contain $ do
safePut isUsed
safePut isChange

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This instance is superfluous, InDb should only be used to tag types defined in core, anything defined by the wallet itself (that we have control ovr) we can just use deriveSafeCopy.

{-------------------------------------------------------------------------------
Wrap core types so that we can make independent serialization decisions
-------------------------------------------------------------------------------}

-- | Wrapped type (with potentially different 'SafeCopy' instance)
-- | Wrapped type (with potentially different 'SC' instance)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this renaming?

@edsko
Copy link
Contributor

edsko commented Aug 23, 2018

Thanks for this @parsonsmatt ! As agreed, I'll take it from here, let's get this thing done and merged :)

@edsko
Copy link
Contributor

edsko commented Aug 23, 2018

Fixed the tests. Am now going to rebase, then force push.

Also placate hlint, stylish-haskell, stack2nix
@edsko
Copy link
Contributor

edsko commented Aug 23, 2018

I recompiled the InDb module with

{-# OPTIONS_GHC -ddump-ds -ddump-to-file #-}

and grepped for $fSafeCopy to check which instances we are using. Apart from the ones for InDb itself (a lot of those), the only other ones are

safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopy(
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopy(,
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopy[]
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyByteString
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyByteString0
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyInt32
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyInteger
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyMap
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyMaybe
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyText0
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyWord16
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyWord32
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyWord64
safecopy-0.9.4.1:Data.SafeCopy.Instances.$fSafeCopyWord8

In other words, none from core! Perfect :)

@KtorZ KtorZ merged commit 47ab0a5 into develop Aug 23, 2018
@KtorZ
Copy link
Contributor

KtorZ commented Aug 23, 2018

Merging into develop, cherry-picked onto develop-new 👍

@edsko
Copy link
Contributor

edsko commented Aug 23, 2018

Muchas gracias :)

@edsko edsko deleted the rc-cbr-305 branch August 23, 2018 13:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants