sapling getdeps cli build and test support for python 3.12 #964
Commits on Oct 7, 2024
-
Summary: sapling github CI had a failure on fb303 and I wasn't sure if fb303 was itself building ok on github, so lets give fb303 a CI. * add a github action to build fb303 * delete fb303 github side packit config [as it is broken](https://github.com/facebook/fb303/pull/58/checks?check_run_id=31169950255). For folly packit was moved to copr side in D54958528 so I think this is fine. (the underlying problem causing a transient [fb303 break in sapling CI](https://github.com/facebook/sapling/actions/runs/11213566522/job/31166881318?pr=963) was that build/deps/github_hashes/facebook/fbthrift-rev.txt was lagging the code using fbthrift, and thus python libs where not installed) X-link: facebook/fb303#58 X-link: facebookincubator/zstrong#1006 Reviewed By: singhsrb Differential Revision: D63976936
-
run .t tests for getdeps sapling cli build (facebook#963)
Summary: Connect up the sapling *.t tests so that github CI has visible cli test status. To get them to run: * some needed fbpython on path. I included a shim for that in the test makefile target. * test-rust-hooks: Command not found message, added a glob * test-identity.t: add a glob for the sapling version * test-eolfilename.t: output order was unstable, added sorts to make stable * helpers-testrepo.sh: fix assumption that system hg would be able to read test repo, check if its Sapling first. * added a manifest for the hexdump utility some of the tests required * excluded a few remaining tests (see comments in Makefile for reason) * fixed getdeps support for generating actions steps for test only dependencies NB the tests run as "hg". The expectations would need to be updated if we were to run as "sl" X-link: facebookincubator/zstrong#1004 Test Plan: enter ubuntu 22.04 toolbox: `toolbox enter ubuntu-toolbox-22.04 ` make sure system packages are installed `./build/fbcode_builder/getdeps.py install-system-deps --recursive sapling` build sapling (and any depedencieds it requires) `./build/fbcode_builder/getdeps.py build --allow-system-packages --src-dir=. sapling` run the tests. 64 concurrent jobs worked well on my 64GB machine and took 2m10s. ``` ./build/fbcode_builder/getdeps.py test --allow-system-packages --src-dir=. sapling --num-jobs=64 ---------------------------------------------------------------------- Skipped 2 tests (missing feature: Mononoke server available): test-fb-ext-remotefilelog-log.t test-fb-ext-remotefilelog-worker.t Skipped 3 tests (missing feature: running tests with fsmonitor): test-fsmonitor-filemerge.t test-fsmonitor-nonutf8-path.t test-rust-hgevents.t Skipped 46 tests (skipped): test-atexit-epipe.t test-casefolding.t test-check-clang-format.t test-check-code.t test-check-fix-code.t test-check-help.t test-checkserverbookmark.t test-chg.t test-debugbacktrace.t test-debugshell-namespace.t test-eden-clone.t test-eden-commands.t test-eden-du.t test-eden-glob.t test-eden-sparse.t test-eden-watchman-edenapi-glob.t test-eden-watchman-noedenapi-glob.t test-fb-ext-fbconduit.t test-fb-ext-scm-prompt-compat.t test-fb-ext-scm-prompt-git.t test-fb-ext-tweakdefaults-grep.t test-fsmonitor-casefolding.t test-fsmonitor-fallback.t test-fsmonitor-metadata.t test-fsmonitor-refreshclock.t test-fsmonitor-status.t test-fsmonitor-warn-fresh.t test-histedit-reorder.t test-matcher-expand-globs.t test-mutation-infinitepush.t test-non-working-symlink.t test-progress-rust-renderer.t test-progressfile.t test-purge-fsmonitor.t test-remotenames-journal.t test-runlog.t test-rust-status-in-transaction.t test-sparse-track-ignored.t test-status-case-insensitive.t test-status-fresh-instance.t test-status-root-ignored-py.t test-status-watchman.t test-symlink-migration.t test-tracing-under-chg.t test-treestate-fresh-instance.t test-treestate-trackignore.t # Ran 729 tests, 51 skipped, 0 failed. passed on try 0 ``` generate github actions CI (using 16 concurrent jobs to speed them up a little) ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "Sapling CLI Getdeps " --job-file-prefix=sapling-cli-getdeps_ sapling --project-install-prefix sapling:/ --num-jobs=16 ``` debugged actions locally with: ``` act -r -j build -W .github/workflows/sapling-cli-getdeps_linux.yml ``` Differential Revision: D63958737 Pulled By: ahornby
-
sapling getdeps cli build and test support for python 3.12 (facebook#964
) Summary: Ubuntu 24.04 and Fedora 40 are both python 3.12 based. Update sapling's OSS getdeps build to allow build/run/test on python 3.12 while keeping 3.10 support. * distutils is deprecated in python 3.12 stdlib, but fortunately setuptools is pretty similar and includes a vendored distutils, so updated to use setuptools's version and removed deprecated usage like `find_executable` in favor of `shutil.which()`. * added a manifest so we get python3-setuptools installed. * found some tests needed bunzip2, so update manifest for sapling and bz2 to install it * util.py makedate(): datetime.utcfromtimestamp is deprecated in 3.12, so updated to use non-deprecated methods. Tested with test-command-template.t which shows tz offset. * Makefile: disabled a few tests on 3.12 where updating the expectation to run on 3.12 and 3.10 is tricky * tests modified to run on 3.12 and 3.10: * test-import-eol.t: updated for invalid escape sequence warnings * test-install.t: updated to filter out message that appears in different order on 3.12 vs earlier versions * test-sign-commit.t: updated to add --yes for newer gpg that python 3.12 using distros have X-link: facebookincubator/zstrong#1005 Test Plan: enter ubuntu 24.04 toolbox: `toolbox enter ubuntu-toolbox-24.04 ` make sure system packages are installed `./build/fbcode_builder/getdeps.py install-system-deps --recursive sapling` check oss non-thrift build: ``` make oss $ ./sl --version Fri Oct 4 17:14:19 BST 2024 Sapling 4.4.2_20241004_170203_06bd09bd1dd2 (see https://sapling-scm.com/ for more information) ``` build sapling getdepts `./build/fbcode_builder/getdeps.py build --allow-system-packages --no-facebook-internal --src-dir=. sapling --project-install-prefix sapling:/` run the tests locally to validate python 3.12 operation: ``` ./build/fbcode_builder/getdeps.py --allow-system-packages test --src-dir=. sapling --num-jobs=64 --project-install-prefix sapling:/' ``` generate github actions CI to pick up python3-setuptools ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "Sapling CLI Getdeps " --job-file-prefix=sapling-cli-getdeps_ sapling --num-jobs=16 --project-install-prefix sapling:/ ``` github CI Differential Revision: D63958742 Pulled By: ahornby
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.