Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why is it unnecessary for libraries to commit the lock file? #7488

Closed
iainelder opened this issue Feb 8, 2023 · 6 comments · Fixed by #7506
Closed

Why is it unnecessary for libraries to commit the lock file? #7488

iainelder opened this issue Feb 8, 2023 · 6 comments · Fixed by #7506
Labels
kind/question User questions (candidates for conversion to discussion)

Comments

@iainelder
Copy link
Contributor

For libraries it is not necessary to commit the lock file.

Why not?

The preceding paragraph basically says committing the lock file makes it easier to reproduce the same experience for all users of the code.

Why would it be different for libraries?

The botocove package uses Poetry to manage its dependencies. botocove is a library for querying AWS, intended to be used in scripts. We are now wondering whether to keep the lock file or not.

@dimbleby
Copy link
Contributor

dimbleby commented Feb 8, 2023

You committing the lockfile will make no difference at all to the users of your library, who will consume your library via pypi. The metadata on pypi is derived from the requirements in pyproject.toml.

@iainelder
Copy link
Contributor Author

Thanks, @dimbleby . That makes sense.

For us library developers, I think it still makes sense to keep the lockfile for the sake of reproducibility.

Is there something else I'm missing?

@dimbleby
Copy link
Contributor

dimbleby commented Feb 8, 2023

keeping the lockfile locally is a trade-off: it's true that you get reproducibility, but it also means that if some new dependency has been released that breaks things for consumers of your library, you're likely to be the last to find out about it.

I guess either choice is legit, but the usual advice is for libraries not to commit lockfiles.

eg here's Rust saying the same thing about Cargo.lock https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries

@iainelder
Copy link
Contributor Author

Thanks for the Cargo docs reference. That helps a lot because the Cargo.lock file plays the same role.

Your comment, though, is the one that changes my mind:

if some new dependency has been released that breaks things for consumers of your library, you're likely to be the last to find out about it.

For my project I had suggested periodically explicitly updating versions declared in the lock file and in the pre-commit config (another place to specify build dependencies).

But now it seems that by discarding the lockfile, I can get that behavior automatically at least for the lock file.

Thanks for your help.

How can we adapt your comment for the Poetry documentation?

@neersighted neersighted added the kind/question User questions (candidates for conversion to discussion) label Feb 8, 2023
@neersighted
Copy link
Member

You'll see that Poetry does the same thing, incidentally -- we commit our lock files in both poetry-core (consumed as a library only) and Poetry (an application installed with normal version constraints). It keeps our CI reproducible/prevents random changes in other projects from blocking development, but we still have to refresh our lock file regularly and stay up-to-date to make sure we don't break suddenly for users.

It's a balancing act, and that's a hard nuance to capture, but contributions are certainly welcome if you want to try to summarize these thoughts for the docs.

@iainelder
Copy link
Contributor Author

Thanks @neersighted .

It sounds like one way or another periodic maintenance is required to stay on top of breaking changes. Either you can periodically update the lock file, or you can periodically rerun the CI with whatever the latest versions are on the day. Maybe there are other solutions, but I don't want to veer off-topic.

A short discussion of why you might want to omit the lock file would improve the docs. It can summarize the points raised in this issue.

I'll attempt a couple of paragraphs and submit a PR.

@python-poetry python-poetry locked and limited conversation to collaborators Apr 28, 2023
@finswimmer finswimmer converted this issue into discussion #7847 Apr 28, 2023
asmfstatoil referenced this issue in equinor/neqsim-python Apr 12, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
kind/question User questions (candidates for conversion to discussion)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants