Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Puppet python latest version resolving - Prereleases #690

Open
optical-o opened this issue Apr 18, 2024 · 3 comments · May be fixed by #693
Open

Puppet python latest version resolving - Prereleases #690

optical-o opened this issue Apr 18, 2024 · 3 comments · May be fixed by #693

Comments

@optical-o
Copy link

optical-o commented Apr 18, 2024 •
edited by kenyon
Loading

Hello,

There is an issue during the resolution of the latest version from the pip repository:

It takes the last version that is retrieved by the pip. However, it might not be the latest version but a prerelease/development version.

" | sed -nE 's/.*\\(from versions: (.*, )*(.*)\\)/\\2/p'",

Example of the output for a private repository with a prerelease version.

(from versions: 1.0.0, 1.0.1, 1.0.3, 1.0.4, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.2.0, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.4.0, 1.4.1, 1.5.0, 1.6.0, 1.6.1, 1.7.0, 1.8.0, 1.9.0, 1.9.1, 1.9.2, 1.10.0, 1.10.1, 1.10.2, 1.10.3, 1.10.4, 1.10.5, 2.0.0rc1, 2.0.0rc2, 2.0.0rc3)

--pre
Include pre-release and development versions. By default, pip only finds stable versions.
@teluq-pbrideau
Copy link

It actually happens right now with the pip public package:

Puppet notify a change to me at every run:

Notice: /Stage[main]/Profile::Test/Python::Pip[pip]/Exec[pip_install_pip]/returns: executed successfully                             
:~$ pip install pip==notreallyaversion
Defaulting to user installation because normal site-packages is not writeable
ERROR: Ignored the following yanked versions: 20.0, 20.3.2, 21.2
ERROR: Could not find a version that satisfies the requirement pip==notreallyaversion (from versions: 0.2, 0.2.1, 0.3, 0.3.1, 0.4, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.6.3, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 1.2.1, 1.3, 1.3.1, 1.4, 1.4.1, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.1.0, 6.1.1, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.1.0, 7.1.1, 7.1.2, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.1.0, 8.1.1, 8.1.2, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 10.0.0b1, 10.0.0b2, 10.0.0, 10.0.1, 18.0, 18.1, 19.0, 19.0.1, 19.0.2, 19.0.3, 19.1, 19.1.1, 19.2, 19.2.1, 19.2.2, 19.2.3, 19.3, 19.3.1, 20.0.1, 20.0.2, 20.1b1, 20.1, 20.1.1, 20.2b1, 20.2, 20.2.1, 20.2.2, 20.2.3, 20.2.4, 20.3b1, 20.3, 20.3.1, 20.3.3, 20.3.4, 21.0, 21.0.1, 21.1, 21.1.1, 21.1.2, 21.1.3, 21.2.1, 21.2.2, 21.2.3, 21.2.4, 21.3, 21.3.1, 22.0, 22.0.1, 22.0.2, 22.0.3, 22.0.4, 22.1b1, 22.1, 22.1.1, 22.1.2, 22.2, 22.2.1, 22.2.2, 22.3, 22.3.1, 23.0, 23.0.1, 23.1, 23.1.1, 23.1.2, 23.2, 23.2.1, 23.3, 23.3.1, 23.3.2, 24.0, 24.1b1, 24.1b2)
ERROR: No matching distribution found for pip==notreallyaversion

:~$ pip install pip==notreallyaversion 2>&1 | grep -oP "\(from versions: .*\)" | sed -E "s/\(from versions: (.*?, )*(.*)\)/\2/g" | tr -d "[:space:]"
24.1b2

I’ve successfully retreived the right version with this modified command with more pipe (removing rc and b from the output)
The exclusion should probably be a variable taken as parameter of python::pip{} module?

:~$ pip install pip==notreallyaversion 2>&1 | grep -oP "(?<=\(from versions: )[^\)]*" | tr -d "[:space:]" | sed -e 's/,/\n/g' | grep -Ev '(b|rc)' | tail -1
24.0

or a (little) bit cleaner with awk script:

:~$ pip install pip==notreallyaversion 2>&1 | grep -oP "(?<=\(from versions: )[^\)]*" | awk 'BEGIN {RS=", "} {if ($0 !~ /(b|rc)/) {stable[arraylen++]=$0}} END {print stable[arraylen-1] }'
24.0

There is probably a way to get the look-ahead inside the awk command, butI’ll look further into it next week and make a PR with a suggestion.

@optical-o
Copy link
Author

optical-o commented Jun 16, 2024 •
edited
Loading

I think a more proper way would be to use some regex that meets the specification for the version https://packaging.python.org/en/latest/specifications/version-specifiers/#version-specifiers rather than white-labelling of only rc tags. Note that also ('a' and 'c') is allowed.

For example there are also Developmental releases ('dev')

0.9
1.0.dev1
1.0.dev2
1.0.dev3
1.0.dev4
1.0c1
1.0c2
1.0
1.0.post1
1.1.dev1

@teluq-pbrideau
Copy link

Thanks @optical-o for the source reference.
I see no mention of c in this reference, only rc: [N!]N(.N)*[{a|b|rc}N][.postN][.devN]

Should we exclude post release, as it is «strongly discouraged» to include bug fixes in these? Should we be safe and include them?

The use of post-releases to publish maintenance releases containing actual bug fixes is strongly discouraged. In general, it is better to use a longer release number and increment the final component for each maintenance release.

Also, i’ll use sed to extract versions instead of grep with lookahead:

:~$ pip install pip==notreallyaversion 2>&1 | sed -nE 's/.*\(from versions: ([^\)]*)\)/\1/p' | awk 'BEGIN {RS=", "} {if ($0 !~ /(a|b|rc|dev)/) {stable[arraylen++]=$0}} END {print stable[arraylen-1] }'

@teluq-pbrideau teluq-pbrideau linked a pull request Jun 17, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix update for pre-release version teluq-pbrideau/puppet-python
2 participants
@optical-o @teluq-pbrideau