Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration 11-12: migrate CIDsv1 to raw multihashes #95

Merged
merged 13 commits into from
Nov 17, 2021
Merged

Migration 11-12: migrate CIDsv1 to raw multihashes #95

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
5da50d9
Import migration from old branch
hsanjuan Apr 20, 2021
89c5b50
Update sharness simple migration tests to latest migration
hsanjuan Apr 20, 2021
848595b
Fix go.mod/sum and force newer dependencies
hsanjuan Apr 20, 2021
cf67de4
11-12: Track if the raw mhash already existed before migrating
hsanjuan Apr 20, 2021
fc971af
11-12: Ensure the pinset/mfs trees are migrated back
hsanjuan Apr 22, 2021
a1fe736
Do not unswap Cidv0s in MFS/pinset tree during reversions.
hsanjuan Apr 22, 2021
e3dd7b8
Migration 11-12: Add a test
hsanjuan Apr 26, 2021
6aadb8c
Attempt to enable running tests
hsanjuan Apr 26, 2021
e82e5ce
Fix makefile and previous migration to actually be able to run tests
hsanjuan Apr 26, 2021
2ea97fd
Let tests think we are still in v11 as otherwise they fail
hsanjuan Apr 26, 2021
b059b50
Migration 11-12: vendor dependencies
hsanjuan Apr 26, 2021
bc7f546
Add more comments and cleanup code
hsanjuan Jun 14, 2021
f91ca2d
Merge pull request #131 from ipfs/feat/repo-migration-raw-multihashes…
hsanjuan Nov 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@ test_go: $(shell ls -d fs-repo-*-to-* | sed -e 's/fs-repo/test_go.fs-repo/')

test_go.%: MIGRATION=$*
test_go.%:
@cd $(MIGRATION) && go test -mod=vendor
@cd $(MIGRATION)/migration && go test -mod=vendor
8 changes: 4 additions & 4 deletions fs-repo-10-to-11/migration/migration.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ import (
"github.com/ipfs/go-blockservice"
"github.com/ipfs/go-datastore"
"github.com/ipfs/go-filestore"
"github.com/ipfs/go-ipfs-blockstore"
"github.com/ipfs/go-ipfs-exchange-offline"
blockstore "github.com/ipfs/go-ipfs-blockstore"
offline "github.com/ipfs/go-ipfs-exchange-offline"
"github.com/ipfs/go-ipfs-pinner/pinconv"
"github.com/ipfs/go-ipfs/plugin/loader"
"github.com/ipfs/go-ipfs/repo"
"github.com/ipfs/go-ipfs/repo/fsrepo"
"github.com/ipfs/go-ipfs/repo/fsrepo/migrations"
"github.com/ipfs/go-ipld-format"
format "github.com/ipfs/go-ipld-format"
"github.com/ipfs/go-merkledag"

migrate "github.com/ipfs/fs-repo-migrations/tools/go-migrate"
Expand Down Expand Up @@ -88,7 +88,7 @@ func (m Migration) Apply(opts migrate.Options) error {

err = migrations.WriteRepoVersion(opts.Path, toVer)
if err != nil {
return fmt.Errorf("failed to update version file to %s: %v", toVer, err)
return fmt.Errorf("failed to update version file to %d: %v", toVer, err)
}

log.Print("updated version file")
Expand Down
7 changes: 7 additions & 0 deletions fs-repo-11-to-12/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
.PHONY: build clean

build:
go build -mod=vendor

clean:
go clean
17 changes: 17 additions & 0 deletions fs-repo-11-to-12/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
module github.com/ipfs/fs-repo-migrations/fs-repo-11-to-12

go 1.15

require (
github.com/hsanjuan/ipfs-lite v1.1.19
github.com/ipfs/fs-repo-migrations v1.7.1
github.com/ipfs/go-cid v0.0.7
github.com/ipfs/go-datastore v0.4.5
github.com/ipfs/go-filestore v1.0.0
github.com/ipfs/go-ipfs v0.8.0
github.com/ipfs/go-ipfs-blockstore v1.0.3 // indirect
github.com/ipfs/go-ipfs-ds-help v1.0.0
github.com/ipfs/go-ipfs-pinner v0.1.1
github.com/ipfs/go-ipld-format v0.2.0
github.com/otiai10/copy v1.5.1
)
1,376 changes: 1,376 additions & 0 deletions fs-repo-11-to-12/go.sum
View file
Open in desktop

Large diffs are not rendered by default.

11 changes: 11 additions & 0 deletions fs-repo-11-to-12/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package main

import (
mg11 "github.com/ipfs/fs-repo-migrations/fs-repo-11-to-12/migration"
migrate "github.com/ipfs/fs-repo-migrations/go-migrate"
)

func main() {
m := mg11.Migration{}
migrate.Main(&m)
}
25 changes: 25 additions & 0 deletions fs-repo-11-to-12/migration/generate_repotest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

# This script can be used to generate the repotest folder that is used
# to test the migration, using go-ipfs v0.8.0.

export IPFS_PATH=repotest

mkdir a
echo "file1" > a/file1
mkdir b
echo "file2" > b/file2
mkdir c
echo "file3" > c/file3

ipfs init

# A: add with both v0 and v1
ipfs add -r --pin=false a
ipfs add -r --cid-version=1 --raw-leaves=false --pin=false a

# B: add with v1
ipfs add -r --cid-version=1 --raw-leaves=false --pin=false b

# C: add with v0
ipfs add -r --pin=false c
Loading