feat: add a tar toolchain #468

alexeagle · 2023-07-07T05:05:27Z

This provides an alternative to the pkg_tar rule from rules_pkg. That has the advantage that we can handle runfiles correctly from the outset and be a better recommendation for rules_oci users.

Fixes #470

Type of change

New feature or functionality (change which adds functionality)

For changes visible to end-users

Relevant documentation has been updated
Suggested release notes are provided below:

There is now a tar rule to package files into tar archives. Users of pkg_tar may want to consider this as a simpler alternative. See tar.md

Test plan

New test cases added

thesayyn · 2023-09-29T22:51:26Z

ah, https://man.freebsd.org/cgi/man.cgi?mtree(8)
https://man.freebsd.org/cgi/man.cgi?query=bsdtar&sektion=1&format=html#EXAMPLES

@thesayyn

@thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8)

Also allow arbitrary args

alexeagle · 2023-10-01T20:54:52Z

@kormide @thesayyn should we land this on main or hold it for the 2.0 release so there's a compelling new feature to go along with the major release?

lib/private/tar_toolchain.bzl

lib/tar.bzl

kormide · 2023-10-02T04:15:52Z

@kormide @thesayyn should we land this on main or hold it for the 2.0 release so there's a compelling new feature to go along with the major release?

I think that's a good idea. Unless rules_oci needs it sooner.

alexeagle · 2023-10-02T18:43:26Z

rules_oci users have been waiting 9 months, so a couple more weeks is fine.

That also lets us give tar some testing as a non-public API while we have a 2.0-beta series of releases, only committing to it for the first RC

@thesayyn

* feat: add a BSD tar toolchain @thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8) * show example of mtree usage * feat: introduce tar rule * cleanup and get test passing * more cleanup * chore: add support for compress flags * chore: add docs * chore: add docs * feat: implement linux bsdtar toolchain (#566) * chore: improve target naming * WIP: args * feat: generate mtree spec Also allow arbitrary args * refactor: mtree is required * refactor: style nits * fix: support mix of source and generated artifacts * feat: demonstrate strip_prefix * chore: regen docs * fix: make host toolchain a fallback toolchain * fix: include libarchive13.so when installing BSD tar * chore: buildifier * fix: aarch64 cpu constraint * fix(ci): include libarchive13.so when running tar * chore: add libnettle * refactor: inputs mutated less * refactor: remove unneeded substitution arg * refactor: don't advertise unsupported modes * fix: hack enough to make it run on my machine * chore: dynamic libraries included in sh_binary under toolchain * make sh_binary work * refactor: drop arm64 for now * fix toolchain * fix test * chore: improve test naming scheme --------- Co-authored-by: Sahin Yort <thesayyn@gmail.com>

@thesayyn

* feat: add a BSD tar toolchain @thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8) * show example of mtree usage * feat: introduce tar rule * cleanup and get test passing * more cleanup * chore: add support for compress flags * chore: add docs * chore: add docs * feat: implement linux bsdtar toolchain (#566) * chore: improve target naming * WIP: args * feat: generate mtree spec Also allow arbitrary args * refactor: mtree is required * refactor: style nits * fix: support mix of source and generated artifacts * feat: demonstrate strip_prefix * chore: regen docs * fix: make host toolchain a fallback toolchain * fix: include libarchive13.so when installing BSD tar * chore: buildifier * fix: aarch64 cpu constraint * fix(ci): include libarchive13.so when running tar * chore: add libnettle * refactor: inputs mutated less * refactor: remove unneeded substitution arg * refactor: don't advertise unsupported modes * fix: hack enough to make it run on my machine * chore: dynamic libraries included in sh_binary under toolchain * make sh_binary work * refactor: drop arm64 for now * fix toolchain * fix test * chore: improve test naming scheme --------- Co-authored-by: Sahin Yort <thesayyn@gmail.com>

@thesayyn

* feat: add a BSD tar toolchain @thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8) * show example of mtree usage * feat: introduce tar rule * cleanup and get test passing * more cleanup * chore: add support for compress flags * chore: add docs * chore: add docs * feat: implement linux bsdtar toolchain (#566) * chore: improve target naming * WIP: args * feat: generate mtree spec Also allow arbitrary args * refactor: mtree is required * refactor: style nits * fix: support mix of source and generated artifacts * feat: demonstrate strip_prefix * chore: regen docs * fix: make host toolchain a fallback toolchain * fix: include libarchive13.so when installing BSD tar * chore: buildifier * fix: aarch64 cpu constraint * fix(ci): include libarchive13.so when running tar * chore: add libnettle * refactor: inputs mutated less * refactor: remove unneeded substitution arg * refactor: don't advertise unsupported modes * fix: hack enough to make it run on my machine * chore: dynamic libraries included in sh_binary under toolchain * make sh_binary work * refactor: drop arm64 for now * fix toolchain * fix test * chore: improve test naming scheme --------- Co-authored-by: Sahin Yort <thesayyn@gmail.com>

@thesayyn

* feat: add a BSD tar toolchain @thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8) * show example of mtree usage * feat: introduce tar rule * cleanup and get test passing * more cleanup * chore: add support for compress flags * chore: add docs * chore: add docs * feat: implement linux bsdtar toolchain (#566) * chore: improve target naming * WIP: args * feat: generate mtree spec Also allow arbitrary args * refactor: mtree is required * refactor: style nits * fix: support mix of source and generated artifacts * feat: demonstrate strip_prefix * chore: regen docs * fix: make host toolchain a fallback toolchain * fix: include libarchive13.so when installing BSD tar * chore: buildifier * fix: aarch64 cpu constraint * fix(ci): include libarchive13.so when running tar * chore: add libnettle * refactor: inputs mutated less * refactor: remove unneeded substitution arg * refactor: don't advertise unsupported modes * fix: hack enough to make it run on my machine * chore: dynamic libraries included in sh_binary under toolchain * make sh_binary work * refactor: drop arm64 for now * fix toolchain * fix test * chore: improve test naming scheme --------- Co-authored-by: Sahin Yort <thesayyn@gmail.com>

@thesayyn

* feat: add a BSD tar toolchain @thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8) * show example of mtree usage * feat: introduce tar rule * cleanup and get test passing * more cleanup * chore: add support for compress flags * chore: add docs * chore: add docs * feat: implement linux bsdtar toolchain (#566) * chore: improve target naming * WIP: args * feat: generate mtree spec Also allow arbitrary args * refactor: mtree is required * refactor: style nits * fix: support mix of source and generated artifacts * feat: demonstrate strip_prefix * chore: regen docs * fix: make host toolchain a fallback toolchain * fix: include libarchive13.so when installing BSD tar * chore: buildifier * fix: aarch64 cpu constraint * fix(ci): include libarchive13.so when running tar * chore: add libnettle * refactor: inputs mutated less * refactor: remove unneeded substitution arg * refactor: don't advertise unsupported modes * fix: hack enough to make it run on my machine * chore: dynamic libraries included in sh_binary under toolchain * make sh_binary work * refactor: drop arm64 for now * fix toolchain * fix test * chore: improve test naming scheme --------- Co-authored-by: Sahin Yort <thesayyn@gmail.com>

@thesayyn

* feat: add a BSD tar toolchain @thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8) * show example of mtree usage * feat: introduce tar rule * cleanup and get test passing * more cleanup * chore: add support for compress flags * chore: add docs * chore: add docs * feat: implement linux bsdtar toolchain (#566) * chore: improve target naming * WIP: args * feat: generate mtree spec Also allow arbitrary args * refactor: mtree is required * refactor: style nits * fix: support mix of source and generated artifacts * feat: demonstrate strip_prefix * chore: regen docs * fix: make host toolchain a fallback toolchain * fix: include libarchive13.so when installing BSD tar * chore: buildifier * fix: aarch64 cpu constraint * fix(ci): include libarchive13.so when running tar * chore: add libnettle * refactor: inputs mutated less * refactor: remove unneeded substitution arg * refactor: don't advertise unsupported modes * fix: hack enough to make it run on my machine * chore: dynamic libraries included in sh_binary under toolchain * make sh_binary work * refactor: drop arm64 for now * fix toolchain * fix test * chore: improve test naming scheme --------- Co-authored-by: Sahin Yort <thesayyn@gmail.com>

alexeagle requested a review from thesayyn July 7, 2023 05:05

feat: add a BSD tar toolchain

858e7a8

@thesayyn discovered that it has a feature which should make it a drop-in replacement for pkg_tar including fine-grained file permissions and symlinks: https://man.freebsd.org/cgi/man.cgi?mtree(8)

alexeagle force-pushed the tar branch from 3e9060f to 858e7a8 Compare September 29, 2023 23:30

show example of mtree usage

1c31008

alexeagle changed the title ~~feat: add a GNU tar toolchain~~ feat: add a BSD tar toolchain Sep 30, 2023

alexeagle and others added 15 commits September 29, 2023 18:37

feat: introduce tar rule

0aa8530

cleanup and get test passing

6df4e44

more cleanup

f626c69

chore: add support for compress flags

6515fb0

chore: add docs

4a8ce77

chore: add docs

b3952e1

feat: implement linux bsdtar toolchain (#566)

959c935

chore: improve target naming

0a7ba0f

WIP: args

21c2a61

feat: generate mtree spec

a03cd85

Also allow arbitrary args

refactor: mtree is required

82bdf83

refactor: style nits

ca21629

fix: support mix of source and generated artifacts

bec2335

feat: demonstrate strip_prefix

2f2a43c

chore: regen docs

5cd96f4

alexeagle requested a review from kormide October 1, 2023 20:54

alexeagle marked this pull request as ready for review October 1, 2023 20:54

thesayyn and others added 2 commits October 1, 2023 15:11

fix: make host toolchain a fallback toolchain

61b0d38

fix: include libarchive13.so when installing BSD tar

c8357b2

alexeagle changed the title ~~feat: add a BSD tar toolchain~~ feat: add a tar toolchain Oct 1, 2023

chore: buildifier

419468c

kormide reviewed Oct 2, 2023

View reviewed changes

lib/private/tar_toolchain.bzl Outdated Show resolved Hide resolved

kormide reviewed Oct 2, 2023

View reviewed changes

lib/tar.bzl Show resolved Hide resolved

fix: aarch64 cpu constraint

2a131de

alexeagle changed the base branch from main to 2.x October 2, 2023 18:42

kormide approved these changes Oct 2, 2023

View reviewed changes

This was referenced Oct 2, 2023

oci_tarball does not produce consistent tars for the same inputs bazel-contrib/rules_oci#328

Closed

oci_tarball: set time, user and group to reproducible values for Linux bazel-contrib/rules_oci#381

Closed

alexeagle added 4 commits October 2, 2023 16:17

fix(ci): include libarchive13.so when running tar

65d9505

chore: add libnettle

1afc20c

refactor: inputs mutated less

a4e8bc0

refactor: remove unneeded substitution arg

a12fcf9

alexeagle force-pushed the 2.x branch from d993422 to e42f358 Compare October 2, 2023 23:36

alexeagle added 3 commits October 2, 2023 16:53

refactor: don't advertise unsupported modes

01e3c51

fix: hack enough to make it run on my machine

528a16c

chore: dynamic libraries included in sh_binary under toolchain

95f6add

alexeagle added this to the 2.0 milestone Oct 3, 2023

thesayyn and others added 5 commits October 3, 2023 11:03

make sh_binary work

d57e25a

refactor: drop arm64 for now

db83d35

fix toolchain

8166519

fix test

7f18ac7

chore: improve test naming scheme

4b246f4

alexeagle merged commit bf2ea44 into 2.x Oct 3, 2023
6 checks passed

alexeagle deleted the tar branch October 3, 2023 20:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add a tar toolchain #468

feat: add a tar toolchain #468

alexeagle commented Jul 7, 2023 •

edited

Loading

thesayyn commented Sep 29, 2023

alexeagle commented Oct 1, 2023

kormide commented Oct 2, 2023

alexeagle commented Oct 2, 2023

feat: add a tar toolchain #468

feat: add a tar toolchain #468

Conversation

alexeagle commented Jul 7, 2023 • edited Loading

Type of change

Test plan

thesayyn commented Sep 29, 2023

alexeagle commented Oct 1, 2023

kormide commented Oct 2, 2023

alexeagle commented Oct 2, 2023

alexeagle commented Jul 7, 2023 •

edited

Loading