Tezos release 7.0~rc1 #16215

pirbo · 2020-04-16T17:28:21Z

We can finally release software to participate to the Tezos blockchain in opam "central" repository (up to now, binaries were specific to a specific "chain"). 🎉

For now, we follow the policy to add the constraint { = version } in between all the packages part of the same tar.bz2 but we're not sure about that and we welcome feedback...

An other question: is it OK to have "meta-package" in the opam-repository? In that case, we would add a package tezos with nothing but dependencies to all the binaries (tezos-node, tezos-client, tezos-{accuser,baker,endorser}-006-PsCARTHA) and optional dependencies.

avsm · 2020-04-16T17:39:21Z

Glad to have you here!

There's a small bug in the generation scripts I think:

tezos-accuser-006-PsCARTHA/tezos-accuser-006-PsCARTHA.7.0~rc1/tezos-accuser-006-PsCARTHA.7.0~beta/

You should only have one level of versioned directory there.

It's fine (and encouraged) to set =version for the packages generated from the same repo, as that's how they will be tested together.

A tezos metapackage is just fine, and a good user experience (opam install tezos). Feel free to add that to this PR.

camelus · 2020-04-16T18:04:56Z

Commit: 321bff7

@pirbo has posted 21 contributions.

🌤️ opam-lint warnings `321bff7`

tezos-embedded-protocol-005-PsBabyM1.7.0~rc1 has some warnings:
- warning 41: Some packages are mentioned in package scripts or features, but there is no dependency or depopt toward them: "tezos-protocol-compiler"
tezos-lmdb.7.0~rc1 has some warnings:
- warning 35: Missing field 'homepage'
These packages passed lint tests: ledgerwallet-tezos.0.1.0, ledgerwallet.0.1.0, resto-cohttp-client.0.4, resto-cohttp-server.0.4, resto-directory.0.4, tezos-accuser-006-PsCARTHA.7.0~rc1, tezos-baker-006-PsCARTHA.7.0~rc1, tezos-baking-006-PsCARTHA-commands.7.0~rc1, tezos-baking-006-PsCARTHA.7.0~rc1, tezos-baking-alpha-commands.7.0~rc1, tezos-baking-alpha.7.0~rc1, tezos-base.7.0~rc1, tezos-clic.7.0~rc1, tezos-client-000-Ps9mPmXa.7.0~rc1, tezos-client-001-PtCJ7pwo-commands.7.0~rc1, tezos-client-001-PtCJ7pwo.7.0~rc1, tezos-client-002-PsYLVpVv-commands.7.0~rc1, tezos-client-002-PsYLVpVv.7.0~rc1, tezos-client-003-PsddFKi3-commands.7.0~rc1, tezos-client-003-PsddFKi3.7.0~rc1, tezos-client-004-Pt24m4xi-commands.7.0~rc1, tezos-client-004-Pt24m4xi.7.0~rc1, tezos-client-005-PsBabyM1-commands.7.0~rc1, tezos-client-005-PsBabyM1.7.0~rc1, tezos-client-006-PsCARTHA-commands.7.0~rc1, tezos-client-006-PsCARTHA.7.0~rc1, tezos-client-alpha-commands.7.0~rc1, tezos-client-alpha.7.0~rc1, tezos-client-base-unix.7.0~rc1, tezos-client-base.7.0~rc1, tezos-client-commands.7.0~rc1, tezos-client-demo-counter.7.0~rc1, tezos-client-genesis-carthagenet.7.0~rc1, tezos-client-genesis.7.0~rc1, tezos-client.7.0~rc1, tezos-codec.7.0~rc1, tezos-crypto.7.0~rc1, tezos-embedded-protocol-000-Ps9mPmXa.7.0~rc1, tezos-embedded-protocol-001-PtCJ7pwo.7.0~rc1, tezos-embedded-protocol-002-PsYLVpVv.7.0~rc1, tezos-embedded-protocol-003-PsddFKi3.7.0~rc1, tezos-embedded-protocol-004-Pt24m4xi.7.0~rc1, tezos-embedded-protocol-005-PsBABY5H.7.0~rc1, tezos-embedded-protocol-006-PsCARTHA.7.0~rc1, tezos-embedded-protocol-alpha.7.0~rc1, tezos-embedded-protocol-demo-counter.7.0~rc1, tezos-embedded-protocol-demo-noops.7.0~rc1, tezos-embedded-protocol-genesis-carthagenet.7.0~rc1, tezos-embedded-protocol-genesis.7.0~rc1, tezos-endorser-006-PsCARTHA.7.0~rc1, tezos-error-monad.7.0~rc1, tezos-event-logging.7.0~rc1, tezos-mempool-006-PsCARTHA.7.0~rc1, tezos-micheline.7.0~rc1, tezos-mockup-commands.7.0~rc1, tezos-mockup-registration.7.0~rc1, tezos-mockup.7.0~rc1, tezos-node.7.0~rc1, tezos-p2p-services.7.0~rc1, tezos-p2p.7.0~rc1, tezos-protocol-000-Ps9mPmXa.7.0~rc1, tezos-protocol-001-PtCJ7pwo.7.0~rc1, tezos-protocol-002-PsYLVpVv.7.0~rc1, tezos-protocol-003-PsddFKi3.7.0~rc1, tezos-protocol-004-Pt24m4xi.7.0~rc1, tezos-protocol-005-PsBABY5H.7.0~rc1, tezos-protocol-005-PsBabyM1.7.0~rc1, tezos-protocol-006-PsCARTHA-parameters.7.0~rc1, tezos-protocol-006-PsCARTHA.7.0~rc1, tezos-protocol-alpha-parameters.7.0~rc1, tezos-protocol-alpha.7.0~rc1, tezos-protocol-compiler.7.0~rc1, tezos-protocol-demo-counter.7.0~rc1, tezos-protocol-demo-noops.7.0~rc1, tezos-protocol-environment-sigs.7.0~rc1, tezos-protocol-environment.7.0~rc1, tezos-protocol-genesis-carthagenet.7.0~rc1, tezos-protocol-genesis.7.0~rc1, tezos-protocol-updater.7.0~rc1, tezos-requester.7.0~rc1, tezos-rpc-http-client-unix.7.0~rc1, tezos-rpc-http-client.7.0~rc1, tezos-rpc-http-server.7.0~rc1, tezos-rpc-http.7.0~rc1, tezos-rpc.7.0~rc1, tezos-shell-context.7.0~rc1, tezos-shell-services.7.0~rc1, tezos-shell.7.0~rc1, tezos-signer-backends.7.0~rc1, tezos-signer-services.7.0~rc1, tezos-signer.7.0~rc1, tezos-stdlib-unix.7.0~rc1, tezos-stdlib.7.0~rc1, tezos-storage.7.0~rc1, tezos-tooling.7.0~rc1, tezos-validation.7.0~rc1, tezos-validator.7.0~rc1, tezos-version.7.0~rc1, tezos-workers.7.0~rc1, tezos.7.0~rc1

☀️ Installability check (+97)

new installable packages (97): tezos.7.0~rc1 tezos-accuser-006-PsCARTHA.7.0~rc1 tezos-baker-006-PsCARTHA.7.0~rc1 tezos-baking-006-PsCARTHA.7.0~rc1 tezos-baking-006-PsCARTHA-commands.7.0~rc1 tezos-baking-alpha.7.0~rc1 tezos-baking-alpha-commands.7.0~rc1 tezos-base.7.0~rc1 tezos-clic.7.0~rc1 tezos-client.7.0~rc1 tezos-client-000-Ps9mPmXa.7.0~rc1 tezos-client-001-PtCJ7pwo.7.0~rc1 tezos-client-001-PtCJ7pwo-commands.7.0~rc1 tezos-client-002-PsYLVpVv.7.0~rc1 tezos-client-002-PsYLVpVv-commands.7.0~rc1 tezos-client-003-PsddFKi3.7.0~rc1 tezos-client-003-PsddFKi3-commands.7.0~rc1 tezos-client-004-Pt24m4xi.7.0~rc1 tezos-client-004-Pt24m4xi-commands.7.0~rc1 tezos-client-005-PsBabyM1.7.0~rc1 tezos-client-005-PsBabyM1-commands.7.0~rc1 tezos-client-006-PsCARTHA.7.0~rc1 tezos-client-006-PsCARTHA-commands.7.0~rc1 tezos-client-alpha.7.0~rc1 tezos-client-alpha-commands.7.0~rc1 tezos-client-base.7.0~rc1 tezos-client-base-unix.7.0~rc1 tezos-client-commands.7.0~rc1 tezos-client-demo-counter.7.0~rc1 tezos-client-genesis.7.0~rc1 tezos-client-genesis-carthagenet.7.0~rc1 tezos-codec.7.0~rc1 tezos-crypto.7.0~rc1 tezos-embedded-protocol-000-Ps9mPmXa.7.0~rc1 tezos-embedded-protocol-001-PtCJ7pwo.7.0~rc1 tezos-embedded-protocol-002-PsYLVpVv.7.0~rc1 tezos-embedded-protocol-003-PsddFKi3.7.0~rc1 tezos-embedded-protocol-004-Pt24m4xi.7.0~rc1 tezos-embedded-protocol-005-PsBABY5H.7.0~rc1 tezos-embedded-protocol-005-PsBabyM1.7.0~rc1 tezos-embedded-protocol-006-PsCARTHA.7.0~rc1 tezos-embedded-protocol-alpha.7.0~rc1 tezos-embedded-protocol-demo-counter.7.0~rc1 tezos-embedded-protocol-demo-noops.7.0~rc1 tezos-embedded-protocol-genesis.7.0~rc1 tezos-embedded-protocol-genesis-carthagenet.7.0~rc1 tezos-endorser-006-PsCARTHA.7.0~rc1 tezos-error-monad.7.0~rc1 tezos-event-logging.7.0~rc1 tezos-lmdb.7.0~rc1 tezos-mempool-006-PsCARTHA.7.0~rc1 tezos-micheline.7.0~rc1 tezos-mockup.7.0~rc1 tezos-mockup-commands.7.0~rc1 tezos-mockup-registration.7.0~rc1 tezos-node.7.0~rc1 tezos-p2p.7.0~rc1 tezos-p2p-services.7.0~rc1 tezos-protocol-000-Ps9mPmXa.7.0~rc1 tezos-protocol-001-PtCJ7pwo.7.0~rc1 tezos-protocol-002-PsYLVpVv.7.0~rc1 tezos-protocol-003-PsddFKi3.7.0~rc1 tezos-protocol-004-Pt24m4xi.7.0~rc1 tezos-protocol-005-PsBABY5H.7.0~rc1 tezos-protocol-005-PsBabyM1.7.0~rc1 tezos-protocol-006-PsCARTHA.7.0~rc1 tezos-protocol-006-PsCARTHA-parameters.7.0~rc1 tezos-protocol-alpha.7.0~rc1 tezos-protocol-alpha-parameters.7.0~rc1 tezos-protocol-compiler.7.0~rc1 tezos-protocol-demo-counter.7.0~rc1 tezos-protocol-demo-noops.7.0~rc1 tezos-protocol-environment.7.0~rc1 tezos-protocol-environment-sigs.7.0~rc1 tezos-protocol-genesis.7.0~rc1 tezos-protocol-genesis-carthagenet.7.0~rc1 tezos-protocol-updater.7.0~rc1 tezos-requester.7.0~rc1 tezos-rpc.7.0~rc1 tezos-rpc-http.7.0~rc1 tezos-rpc-http-client.7.0~rc1 tezos-rpc-http-client-unix.7.0~rc1 tezos-rpc-http-server.7.0~rc1 tezos-shell.7.0~rc1 tezos-shell-context.7.0~rc1 tezos-shell-services.7.0~rc1 tezos-signer.7.0~rc1 tezos-signer-backends.7.0~rc1 tezos-signer-services.7.0~rc1 tezos-stdlib.7.0~rc1 tezos-stdlib-unix.7.0~rc1 tezos-storage.7.0~rc1 tezos-tooling.7.0~rc1 tezos-validation.7.0~rc1 tezos-validator.7.0~rc1 tezos-version.7.0~rc1 tezos-workers.7.0~rc1

🌤️ 1 ignored non-opam files:

packages/tezos-lmdb/tezos-lmdb.7.0~rc1/files/tests_use_temp_dir.patch

avsm · 2020-04-17T12:07:49Z

packages/tezos-accuser-006-PsCARTHA/tezos-accuser-006-PsCARTHA.7.0~rc1/opam

+]
+build: [
+  ["dune" "build" "-p" name "-j" jobs]
+  ["mv" "src/proto_006_PsCARTHA/bin_accuser/%{name}%.install" "./"]


It would be good if you could get rid of this line in the packages. In normal operation, dune build -p will build only the install file for the package you have requested. Do you need this explicit install override for some other reason? We may be able to just fix that directly in dune if there's an underlying limitation.

Very good question about general organization of the repository. I always wanted to discuss it more broadly with the community without daring :-)

These mv are necessary because opam files are not at the root of the repository and dune generates .install files next to their corresponding opam file.

In addition of requiring these ugly mv, I think (but I'm not sure I wanted to try) that not having the opam files at the root of the repository breaks opam pin add --dev which is also unfortunate.

On the contrary, as you see the tezos repository defines 97 opam packages so having 97 more files at the root would quite pollute it . Moreover, some part of the code ("economic protocols" in our terminology corresponding to the folders src/proto_*) are supposed to be (un)pluggable (self contained) plugins, if they are there, software is able to understand/follow the corresponding protocol, if they aren't, you have valid binaries but that don't know about these specific rulesets (but can ask them to their "peers"/compile them/dynamically load them on necessity... an other story). If the opam files corresponding to the libraries of the protocol "foobar" has now to be at the root of the repo, adding/removing protocol "foobar" is not as simple as adding/removing folder src/proto_foobar anymore and that's annoying.

So I'm confused about what direction to follow...

Hi @pirbo, glad you were able to release tezos here :-)

The next version of opam should add better support for non-toplevel .opam. See: ocaml/opam#3499 Basically, it allows to add a field subpath in the package description, meaning: this package only requires this fragment of the .tar.gz (or the repository) in order to build. So opam only extracts (or fetch) the required subpath and build directly in the subdirectory. So, from the dune perspective, both.opam and .install are at toplevel.

And, to fix the daily workflow, it adds opam pin add --dev --rec: il will lookup for .opam in sub-directories.

I have no idea whenever the next (feature-)release of opam might happen but, meanwhile, maybe this ugly mv is acceptable. And, maybe, you are the perfect beta-tester for this new feature :-)

Fantastic!
To hear from you first and foremost but also this feature looks to be exactly what we want indeed. No surprise given its original author. :-D
I agree that the mv are acceptable in the meantime.

I'm not quite convinced that the behaviour of dune is quite correct here, though. If the dune build -p works, then the semantics of .install files are that they should be generated at the root of the project to be useful. If you could open up an issue over at https://github.com/ocaml/dune/issues about this, we can take a look at supporting this properly.

We'll be kicking off the opam 2.1 release process in the next weeks, to answer @hnrgrgr's question above. But it would be good to find a solution that'll work with opam 2.0.x as well, since it is widely packaged on OS distributions.

packages/tezos-tooling/tezos-tooling.7.0~rc1/opam

packages/tezos-workers/tezos-workers.7.0~rc1/opam

pirbo · 2020-04-17T16:27:59Z

For the record. I'm investigating the test failure of tezos-lmdb on MacOS in travisCI but on the MacOS computer I have access to, these tests work perfectly so I'm confused and tempted to use the joker "travis-ci specific issue"...

avsm · 2020-04-18T10:08:39Z

This Mac failure is probably sandboxing-related. Are the lmdb tests trying to write somewhere external to the ~/.opam area?

And we mispelled the constraint anyway

pirbo · 2020-04-18T12:21:31Z

Yes, MacOS failure was related to an hardwired reference to /tmp. I've added a commit that fixes it by replacing "/tmp" by Filename.get_temp_dir.

avsm · 2020-04-20T12:23:38Z

Sounds good. I've just triggered a rebuild of this PR to look at the revdeps more closely, as some failed to build due to the volume causing the Docker GC to kick in.

avsm · 2020-04-20T14:39:09Z

This broadly all looks good to go now. There are quite a few spurious CI failures, but I double-checked that ledgerwallet-tezos will install on CentOS (I suspect conf-hiadpi needs OpenSUSE depext metadata to install on OpenSUSE, but this isn't a blocker to merge).

Do you want to push a non-draft tezos.7 set of packages @pirbo?

pirbo · 2020-04-21T07:44:05Z

Thank you very much for all your help.
It has been decided to not merge the rc1 in opam-repository in the end. See you in 2 weeks (hopefully :-) ) for the final v7.0...

avsm · 2020-04-21T09:42:35Z

Look forward to it. Good luck with the release!

pirbo force-pushed the tezos-2020-03 branch 3 times, most recently from c3414d7 to 8b442fa Compare April 17, 2020 07:17

avsm reviewed Apr 17, 2020

View reviewed changes

packages/tezos-tooling/tezos-tooling.7.0~rc1/opam Show resolved Hide resolved

avsm reviewed Apr 17, 2020

View reviewed changes

packages/tezos-workers/tezos-workers.7.0~rc1/opam Outdated Show resolved Hide resolved

pirbo force-pushed the tezos-2020-03 branch from 8b442fa to 6ac61fc Compare April 17, 2020 16:23

ledgerwallet is compatible with opam.1.11

60c7cf5

pirbo added 18 commits April 18, 2020 12:29

Resto is compatible with lwt 5

37ab3a5

And we mispelled the constraint anyway

+tezos-tooling

e0d903a

+tezos-stdlib

8c85cf8

+tezos-error-monad

7c83741

+tezos-event-logging

60f08ca

+tezos-protocol-environment-sigs

7c448f9

+tezos-micheline

77cd042

+tezos-version

1eb244c

+tezos-rpc

7574ca3

+tezos-crypto

758a3de

+tezos-stdlib-unix

84ea026

+tezos-clic

80a19c7

+tezos-base

37b2985

+tezos-protocol-environment

29e6b9d

+tezos-protocol-compiler

846c23c

+tezos-workers

5915134

+tezos-p2p-services

b783c63

+tezos-shell-services

d7aaac7

pirbo added 22 commits April 18, 2020 12:29

+tezos-mockup-commands

4030c44

+tezos-client-base-unix

2fdfa94

+tezos-signer

7beb73d

+tezos-client-protocols Mainnet

d31e862

+tezos-client-genesis-carthagenet

d5d895f

+tezos-client-protocols-commands Mainnet

bd4067a

+tezos-baking-006-PsCARTHA

33756c2

+tezos-baking-006-PsCARTHA-commands

ac37566

+tezos-client

42d665c

+tezos-accuser-006-PsCARTHA

eafcfeb

+tezos-baker-006-PsCARTHA

7b12956

+tezos-endorser-006-PsCARTHA

67284d2

+tezos-protocols development

7de159e

+tezos-embedded-protocols development

32ea7bc

+tezos-protocol-alpha-parameters

3632721

+tezos-client-protocols development

d24fc17

+tezos-client-alpha-commands

88b1e06

+tezos-baking-alpha

2d749fa

+tezos-baking-alpha-commands

83734bf

+tezos-codec

024de59

+tezos meta package

99e7f33

tezos-lmdb: fix tests on travicCI MacOS

321bff7

pirbo force-pushed the tezos-2020-03 branch from 6ac61fc to 321bff7 Compare April 18, 2020 12:18

pirbo mentioned this pull request Apr 21, 2020

Generate install files at root regardless of opam file location ? ocaml/dune#3405

Closed

pirbo closed this Apr 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tezos release 7.0~rc1 #16215

Tezos release 7.0~rc1 #16215

pirbo commented Apr 16, 2020 •

edited

Loading

avsm commented Apr 16, 2020 •

edited

Loading

camelus commented Apr 16, 2020 •

edited

Loading

avsm Apr 17, 2020

pirbo Apr 17, 2020 •

edited

Loading

hnrgrgr Apr 17, 2020 •

edited

Loading

pirbo Apr 17, 2020

avsm Apr 17, 2020

pirbo commented Apr 17, 2020

avsm commented Apr 18, 2020

pirbo commented Apr 18, 2020

avsm commented Apr 20, 2020

avsm commented Apr 20, 2020

pirbo commented Apr 21, 2020

avsm commented Apr 21, 2020

Tezos release 7.0~rc1 #16215

Tezos release 7.0~rc1 #16215

Conversation

pirbo commented Apr 16, 2020 • edited Loading

avsm commented Apr 16, 2020 • edited Loading

camelus commented Apr 16, 2020 • edited Loading

🌤️ opam-lint warnings 321bff7

☀️ Installability check (+97)

🌤️ 1 ignored non-opam files:

avsm Apr 17, 2020

Choose a reason for hiding this comment

pirbo Apr 17, 2020 • edited Loading

Choose a reason for hiding this comment

hnrgrgr Apr 17, 2020 • edited Loading

Choose a reason for hiding this comment

pirbo Apr 17, 2020

Choose a reason for hiding this comment

avsm Apr 17, 2020

Choose a reason for hiding this comment

pirbo commented Apr 17, 2020

avsm commented Apr 18, 2020

pirbo commented Apr 18, 2020

avsm commented Apr 20, 2020

avsm commented Apr 20, 2020

pirbo commented Apr 21, 2020

avsm commented Apr 21, 2020

pirbo commented Apr 16, 2020 •

edited

Loading

avsm commented Apr 16, 2020 •

edited

Loading

camelus commented Apr 16, 2020 •

edited

Loading

🌤️ opam-lint warnings `321bff7`

pirbo Apr 17, 2020 •

edited

Loading

hnrgrgr Apr 17, 2020 •

edited

Loading