Add simplicity options to pkg build system #6274

ocket8888 · 2021-10-13T10:10:59Z

Adds the following options to pkg:

-S suppress outputting "source RPMs"
-s use simple RPM names e.g. traffic_ops.rpm instead of traffic_ops-6.1.0-11637.ec9ff6a6.el8.x86_64.rpm
-L suppress outputting logs in files (no e.g. traffic_ops_build.log)
and the -h option to print help text

Previously, help text was output whenever an unknown option was passed. Now, passing unknown options causes pkg to exit with a failure and print the offending option(s) followed by usage information, while -h causes it to exit with a success after printing usage information.

The hope is to eventually use this to make the Makefile for CDN-in-a-Box much, much simpler, and to enable it to build the RPMs and emplace them properly without access to the repo's git information in situations where it may not be accessible (as is the case in e.g. extracted release tarballs).

Which Traffic Control components are affected by this PR?

documentation

None/all, depending on your point of view. Only build system files were changed.

What is the best way to verify this PR?

Make sure that pkg still works normally when these aren't passed. Specifically, I checked the following:

calling pkg with no arguments and/or options builds all RPMs and SRPMs, with build logs and the normal, long-form RPM filenames
calling pkg with no options while specifying a single project builds only the RPMs and SRPMs for that project, with build logs and the normal, long-form RPM filenames
calling pkg with -h prints usage information and exits with code 0
calling pkg with an unrecognized option prints the option(s) it didn't recognize, followed by usage information, and then exits with a non-zero code
calling pkg with a single project specified and -S option given outputs only an RPM and build log for that project, no SRPM, with the normal, long-form RPM filename
calling pkg with a single project specified and -s option given outputs only the RPM and SRPM for that project, with build logs, but using simple filenames containing no version information, only the project name.
calling pkg with a single project specified and -L option given outputs only the RPM and SRPM for that project, without build logs, and using the normal, long-form RPM filenames
calling pkg with a single project specified and -S and -s given outputs only the RPM and build log for that project, no SRPM, and using a simple RPM filename containing no version information, only the project name.
calling pkg with a single project specified and -S and -L given outputs only an RPM for the given project, no SPRM or build log, and uses the normal, long-form RPM filename
calling pkg with a single project specified and -s and -L given outputs only the RPM and SRPM for the given project, without a build log, and using simple RPM filenames containing no version information, only the project name.
calling pkg with a single project specified and -S, -s, and -L given outputs only an RPM for the given project, without an SRPM or build log, and using a simple RPM file name containing no version information, only the project name.

PR submission checklist

Existing tests are sufficient, as the new behavior is not depended upon as part of the regular, documented, supported build process.
This PR has documentation
This PR has a CHANGELOG.md entry
This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

zrhoffman · 2021-10-13T16:00:53Z

Does this fix #6273?

ocket8888 · 2021-10-13T20:05:42Z

Does this fix #6273?

No, it leaves that behavior unchanged

tcfdev

Verified all functionality works and previous functionality remains.

build/functions.sh

zrhoffman · 2021-10-22T17:56:33Z

I am still reviewing this

zrhoffman

grove, grovetccfg, and traffic_router still include the build number in the RPM filename when -s is specified.
It was undocumented, but previously, --help would print the help text, too. Now, it prints
```
./pkg: illegal option -- -
./pkg: illegal option -- e
```
followed by the help text printed to stderr, with exit code 1.

build/functions.sh

pkg

build/clean_build.sh

pkg

build/functions.sh

ocket8888 · 2021-10-22T21:46:48Z

It was undocumented, but previously, --help would print the help text, too. Now, it prints ...

That's because it was still parsing those as illegal, unrecognized options. You could also see the help text by passing --spaghetti. getopt does not support GNU-style long-options.

zrhoffman · 2021-10-22T22:03:14Z

getopt does not support GNU-style long-options.

Fair enough, if we ever add support for other long options, we can add --help then, too.

zrhoffman

All of the additional changes look good. Besides the flag sort order, the only outstanding review comment is

grove, grovetccfg, and traffic_router still include the build number in the RPM filename when -s is specified.

pkg

ocket8888 · 2021-10-25T19:14:10Z

grove, grovetccfg, and traffic_router still include the build number in the RPM filename when -s is specified.

grove, grovetccfg, and tomcat also were building but not outputting source RPMs. I fixed it so their respective source RPMs are copied to dist (when not disabled).

Adds the -S option to suppress outputting "source RPMs", the -s option to use simple RPM names, the -L option to suppress outputting log s in files, and the -h option to print help text. Previously, help text was output whenever an unknown option was passed. Now, passing unknown options causes pkg to exit with a failure and print the offending option followed by usage information, while -h causes it to exit with a success after printing usage information.

Also fixed grove, grovetccfg, and tomcat not outputting source RPMs

zrhoffman

It looks like -s only applies to RPMs right now. Should it apply to tarballs, too?

apache-trafficcontrol-6.1.0-docs.tar.gz
apache-trafficcontrol-6.1.0.tar.gz

ocket8888 · 2021-10-26T22:35:48Z

It could, for sure. Would be consistent. I just doubt there's a lot of value in that, since the tarball has to be constructed from the source tree anyway, and idk how that could be useful outside of making a release.

zrhoffman · 2021-10-26T23:05:04Z

Okay, maybe someone can make that improvement in a future PR.

zrhoffman

-S omits source RPMs
-s removes version-release-architecture from the RPM filename
-L suppresses logs
-h shows help text
Existing functionality, besides --spaghetti, remains intact

ocket8888 added build related to the build process documentation related to documentation improvement The functionality exists but it could be improved in some way. labels Oct 13, 2021

ocket8888 force-pushed the pkg-simple-packages branch from b8153a6 to 1bce131 Compare October 22, 2021 16:22

tcfdev approved these changes Oct 22, 2021

View reviewed changes

build/functions.sh Outdated Show resolved Hide resolved

zrhoffman requested changes Oct 22, 2021

View reviewed changes

ocket8888 force-pushed the pkg-simple-packages branch 2 times, most recently from 770cf03 to 533443b Compare October 25, 2021 17:39

zrhoffman requested changes Oct 25, 2021

View reviewed changes

pkg Outdated Show resolved Hide resolved

ocket8888 added 15 commits October 26, 2021 07:53

Add new and missing pkg options to docs

002c614

Use auto-numbering footnote for optional docker-compose dependency

29e9684

Use auto-numbering footnote for default projects

6ea3e28

Fix some projects not listed as default

8f158ee

Document optional projects

5897711

Use auto-numbering footnotes for native build dependency notes

0d2e35e

Fix rendering/formatting of lists

9fa2431

Fix -l incorrect format and not respecting -o

904b2c1

Update CHANGELOG

e3190f5

change variable name, use src.rpm instead of .srpm

2aa7956

flags in help-text order

b4a51f6

Use better subshell syntax

a1edbb1

Fix help text indentation

71dba2e

Always export new variables

f0ffd7f

ocket8888 added 4 commits October 26, 2021 07:53

Revert help text to a heredoc

ddbd539

Order options to be sorted in en_US.utf8

c1a1430

Fix grove, grovetccfg, and traffic_router not respecting '-s'

946f9c6

Also fixed grove, grovetccfg, and tomcat not outputting source RPMs

comply with en_US.utf8's weird string collation

b90149b

ocket8888 force-pushed the pkg-simple-packages branch from bea8af2 to b90149b Compare October 26, 2021 14:28

zrhoffman requested changes Oct 26, 2021

View reviewed changes

zrhoffman approved these changes Oct 26, 2021

View reviewed changes

zrhoffman merged commit e2ba960 into apache:master Oct 26, 2021

ocket8888 deleted the pkg-simple-packages branch October 27, 2021 00:34

zrhoffman mentioned this pull request Dec 10, 2021

Set defaults for simplicity environment variables #6420

Merged

4 tasks

shamrickus mentioned this pull request Jul 20, 2022

Fix Pkg Build Order to Mirror docker-compose.yml #6974

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add simplicity options to pkg build system #6274

Add simplicity options to pkg build system #6274

ocket8888 commented Oct 13, 2021 •

edited

Loading

zrhoffman commented Oct 13, 2021

ocket8888 commented Oct 13, 2021

tcfdev left a comment

zrhoffman commented Oct 22, 2021

zrhoffman left a comment

ocket8888 commented Oct 22, 2021 •

edited

Loading

zrhoffman commented Oct 22, 2021

zrhoffman left a comment

ocket8888 commented Oct 25, 2021

zrhoffman left a comment

ocket8888 commented Oct 26, 2021

zrhoffman commented Oct 26, 2021

zrhoffman left a comment

Add simplicity options to pkg build system #6274

Add simplicity options to pkg build system #6274

Conversation

ocket8888 commented Oct 13, 2021 • edited Loading

Which Traffic Control components are affected by this PR?

What is the best way to verify this PR?

PR submission checklist

zrhoffman commented Oct 13, 2021

ocket8888 commented Oct 13, 2021

tcfdev left a comment

Choose a reason for hiding this comment

zrhoffman commented Oct 22, 2021

zrhoffman left a comment

Choose a reason for hiding this comment

ocket8888 commented Oct 22, 2021 • edited Loading

zrhoffman commented Oct 22, 2021

zrhoffman left a comment

Choose a reason for hiding this comment

ocket8888 commented Oct 25, 2021

zrhoffman left a comment

Choose a reason for hiding this comment

ocket8888 commented Oct 26, 2021

zrhoffman commented Oct 26, 2021

zrhoffman left a comment

Choose a reason for hiding this comment

ocket8888 commented Oct 13, 2021 •

edited

Loading

ocket8888 commented Oct 22, 2021 •

edited

Loading