Do not normalize CidV0 to CidV1 - it breaks fil-markets :cryingbear:

A piece of code from a different project ( .storage ) got copied over together
with the deal-on-chain tracking gadget. While in other projects this is an
innocuous and even welcome addition, it is a catastrophic bug any time evergreen
interacts with a deal originally made using CidV0.

The core reason for this lies deeply in the car file format: it is not sensitive
to multibase variations, but it *is* sensitive to cid version. The sequence of
failure events would be:

- Participant retrieves a deal made with a CidV0 root, exports as car
- This car is currently likely to end up as the exact bytes that were in the
  sealed sector, thus will continue satisfying the PieceCid (*)
- The new deal proposal made by Evergreen upgrades the `Label` to Cidv1. At
  this point nothing odd is going on yet - the Label should ideally be correct
  but it is not a must / there is no cheap mechanism to verify this anyway.
- The new deal is satisfied and sealed. Now the dagstore has an entry for a
  CidV1 even though the underlying car ( and thus PieceCid ) refer to CidV0.
- *Another* participant tries to retrieve the deal evergreen made: this is
  where things go sideways. Either the dagstore does byte-level CID equivalence
  and outright says "I do not have this", or (worse) the dag is streamed out
  over graphsync and a new car is assembled with a different Header and first
  block. This in turn means that the subsequent user will not be able to
  arrive at the same PieceCid and will not be able to satisfy their own deal.

Since we currently "quantize" every individual car file, regardless of transfer
method (see * below) the only expedient way to fix this is to remove the
normalization when a CidV0 is encountered, declare all "incorrect" deals
invalid, and get the individual SPs to re-seal the same cars with the "proper"
proposal label in a few days.

The original code and its inclusion in this project were both a singlehanded
product of one Mr Ribasushi. Shame be on his name.

(*) This entire change is a temporary hack to unblock Evergreen, it is by no
means a robust nor desired solution. The Label field has been advisory from
the launch of the network, and there are already plenty of proposals on chain
with malformed/incorrect/missing Label's. Moreover - the assumption that a car
file can be "quantized" and recreated is extremely fragile, and is quickly
becoming outright wrong. The savvier a client the less likely they are to
produce a "depth-first" car, and less likely to use a quantizing transport
like graphsync (e.g. .storage is switching all their filecoin dealmaking to
non-indeterminstic cars by the end of 2022)

In order for programs like Evergreen and later on various DAO/FVM-based renewal
mechanism to work, there must be a way to bulk retrieve a payload by its PieeCid
alone. This will be written up as a feature request for Boost in the next few
weeks.

common/util.go

@@ -2,17 +2,8 @@ package common
 
 import (
 	"regexp"
-
-	"github.com/ipfs/go-cid"
 )
 
-func CidV1(c cid.Cid) cid.Cid {
-	if c.Version() == 1 {
-		return c
-	}
-	return cid.NewCidV1(c.Type(), c.Hash())
-}
-
 const (
 	cidTrimPrefix = 6
 	cidTrimSuffix = 8

cron/trackdeals.go

@@ -252,7 +252,7 @@ var trackDeals = &cli.Command{
 
 			var labelCid *string
 			if lc, err := cid.Parse(d.Proposal.Label); err == nil {
-				lcs := common.CidV1(lc).String()
+				lcs := lc.String()
 				labelCid = &lcs
 			}
 

misc/pg_schema.sql

@@ -50,6 +50,14 @@ AS $$
   SELECT SUBSTRING( $1 FROM 1 FOR 2 ) = 'ba'
 $$;
 
+CREATE OR REPLACE
+  FUNCTION evergreen.valid_cid(TEXT) RETURNS BOOLEAN
+    LANGUAGE sql IMMUTABLE PARALLEL SAFE
+AS $$
+  SELECT ( SUBSTRING( $1 FROM 1 FOR 2 ) = 'ba' OR SUBSTRING( $1 FROM 1 FOR 2 ) = 'Qm' )
+$$;
+
+
 CREATE OR REPLACE
   FUNCTION evergreen.update_entry_timestamp() RETURNS TRIGGER
     LANGUAGE plpgsql
@@ -99,7 +107,7 @@ CREATE INDEX IF NOT EXISTS pieces_padded_size_idx ON evergreen.pieces ( padded_s
 
 CREATE TABLE IF NOT EXISTS evergreen.payloads (
   piece_cid TEXT NOT NULL REFERENCES evergreen.pieces ( piece_cid ),
-  payload_cid TEXT NOT NULL CONSTRAINT valid_rcid CHECK ( evergreen.valid_cid_v1( payload_cid ) ),
+  payload_cid TEXT NOT NULL CONSTRAINT valid_rcid CHECK ( evergreen.valid_cid( payload_cid ) ),
   CONSTRAINT payload_piece UNIQUE ( payload_cid, piece_cid ),
   CONSTRAINT temp_single_root UNIQUE ( piece_cid ),
   meta JSONB
@@ -154,7 +162,7 @@ CREATE TABLE IF NOT EXISTS evergreen.published_deals (
   provider_id TEXT NOT NULL REFERENCES evergreen.providers ( provider_id ),
   client_id TEXT NOT NULL REFERENCES evergreen.clients ( client_id ),
   label BYTEA NOT NULL,
-  decoded_label TEXT CONSTRAINT valid_cid CHECK ( evergreen.valid_cid_v1( decoded_label ) ),
+  decoded_label TEXT CONSTRAINT valid_label_cid CHECK ( evergreen.valid_cid( decoded_label ) ),
   is_filplus BOOL NOT NULL,
   status TEXT NOT NULL,
   status_meta TEXT,