Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Non recursive automake and Debian packaging changes #700

Merged
merged 5 commits into from
Feb 10, 2023
Merged

Non recursive automake and Debian packaging changes #700

merged 5 commits into from
Feb 10, 2023

Conversation

saiarcot895
Copy link
Contributor

@saiarcot895 saiarcot895 commented Oct 18, 2022 •
edited
Loading

This PR brings in three major changes:

  1. Switch to using non-recursive automake. The primary advantage of this is that builds for source files across subdirectories can happen in parallel, which reduces the build time on systems with sufficient cores. On my dev VM, the build time went from about 2:55 to 2:28 for the Bullseye build.
  2. Update the Debian packaging files to a more recent syntax/features supported in Debian Buster (and newer).
  3. Disable building static libraries. The build time dropped to about 1:30 on my dev VM.

@saiarcot895
Switch to using non-recursive automake
c4fa5d2 
This can help increase build parallelization. On my dev VM, this
decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python
2 disabled, -j12). Time improvements may be more noticeable on systems
with more CPU cores.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
@saiarcot895
Update debian packaging
e832de1 
Update to using debhelper compat 12, remove some overrides that are no
longer needed, use automatically-generated dbgsym packages, and disable
the static build.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
@saiarcot895
Always enable all hardening
ff8155d 
Make sure hardened build flags are always used.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
@saiarcot895
Re-add libboost1.71-dev as a dependency, since the Buster slave conta…
1404402 
…iners have that installed instead of just libboost-dev

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
@saiarcot895 saiarcot895 force-pushed the non-recursive-automake branch from 80c3cf2 to 1404402 Compare October 19, 2022 00:19
qiluo-msft
qiluo-msft previously approved these changes Oct 22, 2022
View reviewed changes
Copy link
Contributor

@qiluo-msft qiluo-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please wait for other reviewers.

@saiarcot895 saiarcot895 merged commit 85f3776 into sonic-net:master Feb 10, 2023
@saiarcot895 saiarcot895 deleted the non-recursive-automake branch February 10, 2023 23:19
@saiarcot895 saiarcot895 mentioned this pull request Feb 10, 2023
Closed
AntonHryshchuk added a commit to AntonHryshchuk/sonic-buildimage that referenced this pull request Feb 22, 2023
@AntonHryshchuk
[submodule] Advance sonic-swss-common pointer
597acf5 
Update sonic-swss-common submodule pointer to include the following:
* 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749))
* 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683))
* 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747))
* 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700))

Signed-off-by: AntonHryshchuk <antonh@nvidia.com>
dprital added a commit to dprital/sonic-buildimage that referenced this pull request Feb 23, 2023
@dprital
[submodule] Advance sonic-swss-common pointer
484776c 
Update sonic-swss-common submodule pointer to include the following:
* 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751))
* 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749))
* 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683))
* 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747))
* 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700))

Signed-off-by: dprital <drorp@nvidia.com>
@dprital dprital mentioned this pull request Feb 23, 2023
Merged
8 tasks
lguohan pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Feb 27, 2023
@dprital
[submodule] Advance sonic-swss-common pointer (#13950)
94cfe83 
Update sonic-swss-common submodule pointer to include the following:
* 565ad4b Fix common path issue ([#751](sonic-net/sonic-swss-common#751))
* 3352881 Prevent sonic-db-cli generate core dump ([#749](sonic-net/sonic-swss-common#749))
* 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([#683](sonic-net/sonic-swss-common#683))
* 8b09f90 Update path to sairedis tests ([#747](sonic-net/sonic-swss-common#747))
* 85f3776 Non recursive automake and Debian packaging changes ([#700](sonic-net/sonic-swss-common#700))

Signed-off-by: dprital <drorp@nvidia.com>
saiarcot895 added a commit to saiarcot895/sonic-buildimage that referenced this pull request Mar 7, 2023
@saiarcot895
[submodule] Advance sonic-swss-common pointer
75eee5f 
Update sonic-swss-common submodule pointer to include the following:
* 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751))
* 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749))
* 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683))
* 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747))
* 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700))

This is a reland of sonic-net#13950, with the debug image build fix.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
@saiarcot895 saiarcot895 mentioned this pull request Mar 7, 2023
Merged
8 tasks
AntonHryshchuk added a commit to AntonHryshchuk/sonic-buildimage that referenced this pull request Mar 8, 2023
@AntonHryshchuk
[submodule] Advance sonic-swss-common pointer
106d6ed 
Update sonic-swss-common submodule pointer to include the following:
* 06ffb51 Define ACL_TABLE and ACL_RULE table in STATE_DB ([sonic-net#748](sonic-net/sonic-swss-common#748))
* 1b369ab [ci] Fix apt-get install unable locate package issue. ([sonic-net#753](sonic-net/sonic-swss-common#753))
* 619d4ec Improve unit test for go wrapper ([sonic-net#752](sonic-net/sonic-swss-common#752))
* 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751))
* 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749))
* 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683))
* 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747))
* 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700))

Signed-off-by: AntonHryshchuk <antonh@nvidia.com>
@saiarcot895
Copy link
Contributor Author

Requesting cherry-pick to 202205 and 202211 for space reduction purposes (removal of static libraries).

xumia pushed a commit to xumia/sonic-buildimage-1 that referenced this pull request Mar 10, 2023
@dprital @xumia
[submodule] Advance sonic-swss-common pointer (sonic-net#13950)
8587af0 
Update sonic-swss-common submodule pointer to include the following:
* 565ad4b Fix common path issue ([sonic-net#751](sonic-net/sonic-swss-common#751))
* 3352881 Prevent sonic-db-cli generate core dump ([sonic-net#749](sonic-net/sonic-swss-common#749))
* 43cadec Add ProfileProvider class to support read profile config from PROFILE_DB. ([sonic-net#683](sonic-net/sonic-swss-common#683))
* 8b09f90 Update path to sairedis tests ([sonic-net#747](sonic-net/sonic-swss-common#747))
* 85f3776 Non recursive automake and Debian packaging changes ([sonic-net#700](sonic-net/sonic-swss-common#700))

Signed-off-by: dprital <drorp@nvidia.com>
@yxieca
Copy link
Contributor

yxieca commented Mar 15, 2023

@saiarcot895 please create separate PR for 202205 branch.

StormLiangMS pushed a commit that referenced this pull request Mar 19, 2023
@saiarcot895 @StormLiangMS
Non recursive automake and Debian packaging changes (#700)
7f40cde 
* Switch to using non-recursive automake

This can help increase build parallelization. On my dev VM, this
decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python
2 disabled, -j12). Time improvements may be more noticeable on systems
with more CPU cores.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

* Update debian packaging

Update to using debhelper compat 12, remove some overrides that are no
longer needed, use automatically-generated dbgsym packages, and disable
the static build.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

* Always enable all hardening

Make sure hardened build flags are always used.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

* Re-add libboost1.71-dev as a dependency, since the Buster slave containers have that installed instead of just libboost-dev

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

---------

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
keboliu added a commit to keboliu/sonic-swss-common that referenced this pull request Apr 1, 2023
@keboliu
Revert "Non recursive automake and Debian packaging changes (sonic-ne…
f3157e6 
…t#700)"

This reverts commit 7f40cde.
@saiarcot895 saiarcot895 mentioned this pull request Apr 3, 2023
Merged
8 tasks
saiarcot895 added a commit to saiarcot895/sonic-buildimage that referenced this pull request Apr 3, 2023
@saiarcot895
[submodule] Advance sonic-swss-common pointer
400c143 
Update sonic-swss-common submodule pointer to include the following:

* 6e4daf1 Revamp module build script to make it work for 5.15 on Ubuntu 20.04 (sonic-net/sonic-swss-common#720)
* 7f40cde Non recursive automake and Debian packaging changes (sonic-net/sonic-swss-common#700)

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
saiarcot895 added a commit to saiarcot895/sonic-swss-common that referenced this pull request Apr 5, 2023
@saiarcot895
[202205] Non recursive automake and Debian packaging changes (sonic-n…
0089791 
…et#700)

* Switch to using non-recursive automake

This can help increase build parallelization. On my dev VM, this
decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python
2 disabled, -j12). Time improvements may be more noticeable on systems
with more CPU cores.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

* Update debian packaging

Update to using debhelper compat 12, remove some overrides that are no
longer needed, use automatically-generated dbgsym packages, and disable
the static build.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

* Always enable all hardening

Make sure hardened build flags are always used.

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

* Re-add libboost1.71-dev as a dependency, since the Buster slave containers have that installed instead of just libboost-dev

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>

---------

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
@saiarcot895 saiarcot895 mentioned this pull request Apr 5, 2023
Merged
yxieca pushed a commit that referenced this pull request Apr 6, 2023
@saiarcot895
[202205] Non recursive automake and Debian packaging changes (#700) (#…
55fd28a 
…772)

Cherry-pick of #700

Switch to using non-recursive automake
This can help increase build parallelization. On my dev VM, this decreased the compilation time from 2:55 to about 2:28 (Bullseye, Python 2 disabled, -j12). Time improvements may be more noticeable on systems with more CPU cores.

Update debian packaging
Update to using debhelper compat 12, remove some overrides that are no longer needed, use automatically-generated dbgsym packages, and disable the static build.

Always enable all hardening
Make sure hardened build flags are always used.

Re-add libboost1.71-dev as a dependency, since the Buster slave containers have that installed instead of just libboost-dev
yxieca pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Apr 7, 2023
@saiarcot895
[submodule] Advance sonic-swss-common pointer (#14504)
db8bcad 
Update sonic-swss-common submodule pointer to include the following:

* 6e4daf1 Revamp module build script to make it work for 5.15 on Ubuntu 20.04 (sonic-net/sonic-swss-common#720)
* 7f40cde Non recursive automake and Debian packaging changes (sonic-net/sonic-swss-common#700)

Signed-off-by: Saikrishna Arcot <sarcot@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qiluo-msft qiluo-msft qiluo-msft approved these changes

@liuh-80 liuh-80 Awaiting requested review from liuh-80

@zbud-msft zbud-msft Awaiting requested review from zbud-msft

Assignees
No one assigned
Labels
Cherry Pick Conflict_202205 Included in 202205 Branch Included in 202211 Branch Request for 202205 Branch Request for 202211 Branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@saiarcot895 @yxieca @qiluo-msft @StormLiangMS