Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootstrap.hs doesn't work with stack 2.3.1 (Cannot complete repo information for a non SHA1 commit due to non-reproducibility) #391

Open
blackgnezdo opened this issue Jun 3, 2020 · 5 comments

Comments

@blackgnezdo
Copy link
Collaborator

gnezdo@os-ws:~/s$ git clone https://github.com/google/proto-lens.git
Cloning into 'proto-lens'...
remote: Enumerating objects: 61, done.
remote: Counting objects: 100% (61/61), done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 4590 (delta 9), reused 31 (delta 7), pack-reused 4529
Receiving objects: 100% (4590/4590), 1.42 MiB | 0 bytes/s, done.
Resolving deltas: 100% (2592/2592), done.
gnezdo@os-ws:~/s$ cd ./proto-lens/
gnezdo@os-ws:~/s/proto-lens$ git submodule update --init --recursive
Submodule 'google/protobuf' (https://github.com/google/protobuf) registered for path 'google/protobuf'
Cloning into '/home/gnezdo/s/proto-lens/google/protobuf'...
Submodule path 'google/protobuf': checked out '48cb18e5c419ddd23d9badcfe4e9df7bde1979b2'
Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'google/protobuf/third_party/benchmark'
Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'google/protobuf/third_party/googletest'
Cloning into '/home/gnezdo/s/proto-lens/google/protobuf/third_party/benchmark'...
Cloning into '/home/gnezdo/s/proto-lens/google/protobuf/third_party/googletest'...
Submodule path 'google/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8'
Submodule path 'google/protobuf/third_party/googletest': checked out 'c3bb0ee2a63279a803aaad956b9b26d74bf9e6e2'
gnezdo@os-ws:~/s/proto-lens$ stack --version
Version 2.3.1, Git revision de2a7b694f07de7e6cf17f8c92338c16286b2878 (8103 commits) x86_64 hpack-0.33.0
gnezdo@os-ws:~/s/proto-lens$ runghc bootstrap.hs 
Cannot complete repo information for a non SHA1 commit due to non-reproducibility: Git repo at https://github.com/google/proto-lens, commit master in subdirectory proto-lens
bootstrap.hs: readCreateProcess: stack "--stack-yaml=stack-bootstrap.yaml" "path" "--local-install-root" (exit 1): failed
@jinwoo
Copy link
Member

jinwoo commented Jun 3, 2020

The error message is related to this: commercialhaskell/stack#4882

jinwoo added a commit to jinwoo/proto-lens that referenced this issue Jun 3, 2020
Latest stack doesn't seem to support named commits such as 'master'.

Get the commit sha from `git rev-parse` and append it to the yaml file.

This doesn't fix google#391 but
goes one step further. We still get an error like:

$ stack runghc -- bootstrap.hs
Cloning 0bef8c2 from https://github.com/google/proto-lens
Unsupported tarball from /tmp/with-repo-archive189930/foo.tar: Symbolic link dest not found from proto-lens/proto-lens-imports/google to ../../google/protobuf/src/google, looking for proto-lens/../google/protobuf/src/google.
This may indicate that the source is a git archive which uses git-annex.
See commercialhaskell/stack#4579 for further information.
bootstrap.hs: readCreateProcess: stack "--stack-yaml=stack-bootstrap.yaml" "path" "--local-install-root" (exit 1): failed
@jinwoo
Copy link
Member

jinwoo commented Jun 3, 2020

After #392, we go one step further and get this error:

$ stack runghc -- bootstrap.hs 
Cloning 0bef8c2f3da645f068b8a26ac168c1da41608182 from https://github.com/google/proto-lens
Unsupported tarball from /tmp/with-repo-archive212759/foo.tar: Symbolic link dest not found from proto-lens/proto-lens-imports/google to ../../google/protobuf/src/google, looking for proto-lens/../google/protobuf/src/google.
This may indicate that the source is a git archive which uses git-annex.
See https://github.com/commercialhaskell/stack/issues/4579 for further information.
bootstrap.hs: readCreateProcess: stack "--stack-yaml=stack-bootstrap.yaml" "path" "--local-install-root" (exit 1): failed

Apparently related to commercialhaskell/stack#4579.

@jinwoo
Copy link
Member

jinwoo commented Jun 3, 2020

I tried various things but couldn't figure out how to fix this symbolic link error. The new stack doesn't seem to give an escape hatch for ignoring it. @judah any idea?

@jinwoo jinwoo reopened this Jun 3, 2020
@judah
Copy link
Collaborator

judah commented Jun 3, 2020

Oof, sorry, I forgot about this problem. I encountered it a while ago here:
commercialhaskell/stack#5004 (comment)

Previously I just used stack-1.* when bootstrapping, but that won't work anymore due to the changes to proto-lens-runtime's reexport-modules. I suppose one fairly manual approach would be:

  • Delete package.yaml (so proto-lens-runtime.cabal is the source of truth)
  • Run the bootstrapping script with stack-1.*
  • Add back that package.yaml

That might be worth trying to unblock the release. I'll see if I can think of something better.

@judah
Copy link
Collaborator

judah commented Jun 4, 2020

Actually, currently we're in a good place because we're not making any backwards-incompatible changes to the API. So I was able to regenerate it by removing the git-related logic altogether and using the in-tree packages for bootstrapping. It's not a great long-term solution, but it should work for the upcoming release:

diff --git a/bootstrap.hs b/bootstrap.hs
index 1d55ed4..9a00f37 100755
--- a/bootstrap.hs
+++ b/bootstrap.hs
@@ -47,9 +47,6 @@ main = do
     --    overwriting the previous versions.
     callProcess "git" ["checkout", bootstrapCommit, "--",
                        bootstrapModuleRoot </> "Proto"]
-    [sha] <- lines <$> readProcess "git" ["rev-parse", bootstrapCommit] ""
-    -- Append the bootstrapping commit hash to the yaml file.
-    appendFile bootstrappingYaml ("  commit: " ++ sha ++ "\n")
     [installRoot] <- lines <$> readProcess stack
                     [useBootstrappingYaml, "path", "--local-install-root"] ""
     let protocGenHaskell = installRoot </> "bin/proto-lens-protoc"
diff --git a/stack-bootstrap.yaml b/stack-bootstrap.yaml
index a30c62c..6edfe97 100644
--- a/stack-bootstrap.yaml
+++ b/stack-bootstrap.yaml
@@ -1,15 +1,11 @@
-resolver: lts-12.7
+resolver: lts-15.15
 
 packages:
+- proto-lens
 - proto-lens-protoc
+- proto-lens-runtime
 
 # Build the current HEAD proto-lens-protoc against older revisions of proto-lens
 # and proto-lens-descriptors that are consistent with each other.
 extra-deps:
 - ghc-source-gen-0.4.0.0
-- git: https://github.com/google/proto-lens
-  subdirs:
-  - proto-lens
-  - proto-lens-runtime
-  # A line like below will be appended by bootstrap.hs.
-  # commit: 0bef8c2f3da645f068b8a26ac168c1da41608182

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants