Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build][202111] Support Debian snapshot mirror to improve build stability #14664

Merged
merged 6 commits into from
Apr 18, 2023

Conversation

xumia
Copy link
Collaborator

@xumia xumia commented Apr 14, 2023

Why I did it

Cherry-pick commits from master to support the snapshot based mirror, and fix the code conflicts.

ad162ae [Build] Optimize the version control for Debian packages (#14557)
38c5d7f [Build] Support j2 template for debian sources for docker ptf (#13198)
5e4826e [Ci] Support to use the same snapshot for all platform builds (#13913)
8206925 [Build] Change the default mirror version config file (#13786)
5e4a866 [Build] Support Debian snapshot mirror to improve build stability (#13097)
ac5d89c [Build] Support j2 template for debian sources (#12557)

Work item tracking
  • Microsoft ADO (number only): 18018114

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

@xumia xumia added the Build label Apr 14, 2023
@xumia xumia marked this pull request as draft April 14, 2023 10:19
@xumia xumia force-pushed the support-sn-202111 branch 2 times, most recently from 703ed59 to 1496c86 Compare April 14, 2023 13:49
xumia and others added 6 commits April 17, 2023 08:07
commit 69f74304e2c848d4ca4c7767c7b0b228f5351de6
Author: xumia <59720581+xumia@users.noreply.github.com>
Date:   Wed Nov 9 08:09:53 2022 +0800

    [Build] Support j2 template for debian sources (sonic-net#12557)

    Why I did it
    Unify the Debian mirror sources
    Make easy to upgrade to the next Debian release, not source url code change required.
    Support to customize the Debian mirror sources during the build
    Relative issue: sonic-net#12523
…nic-net#13097)

Why I did it
[Build] Support Debian snapshot mirror to improve build stability

It is to enhance the reproducible build, supports the Debian snapshot mirror. It guarantees all the docker images using the same Debian mirror snapshot and fixes the temporary build failure which is caused by remote Debain mirror indexes changed during the build. It is also to fix the version conflict issue caused by no fixed versions of some of the Debian packages.

How I did it
Add a new feature to support the Debian snapshot mirror.

How to verify it
Why I did it
Change the mirror config file
Use the files/build/versions/default/versions-mirror only when reproducible build enabled.
The config in files/build/versions is only for reproducible build, while snapshot mirror feature does not have the dependency on the reproducible build.

How I did it
Skip the mirror config in files/build/versions/default/versions-mirror if reproducible build not enabled.

How to verify it
…net#13198)

Change to use the sources.list from the file generated from the j2 template
…4557)

Why I did it
Optimize the version control for Debian packages.
Fix sonic-slave-buster/sources.list.amd64 not found display issue, need to generate the file before running the shell command to evaluate the sonic image tag.
When using the snapshot mirror, it is not necessary to update the version file based on the base image. It will reduce the version dependency issue, when an image is not run when freezing the version.

How I did it
Not to update the version file when snapshot mirror enabled.

How to verify it
@xumia xumia marked this pull request as ready for review April 18, 2023 02:28
@xumia xumia requested a review from judyjoseph April 18, 2023 02:29
@xumia xumia merged commit 40780c0 into sonic-net:202111 Apr 18, 2023
@xumia xumia deleted the support-sn-202111 branch April 18, 2023 06:32
@nmoray
Copy link
Contributor

nmoray commented May 5, 2023

@xumia: I tried patching this PR, but the build (202111) is failing at the packaging stage with following error. As initramfs-tools-core is failing, other dependent installations are also failing.

[INFO] Install SONiC linux kernel image

  • sudo dpkg --root=./fsroot-innovium -i target/debs/bullseye/initramfs-tools-core_0.140_all.deb
    Selecting previously unselected package initramfs-tools-core.
    (Reading database ... 6806 files and directories currently installed.)
    Preparing to unpack .../initramfs-tools-core_0.140_all.deb ...
    Unpacking initramfs-tools-core (0.140) ...
    dpkg: dependency problems prevent configuration of initramfs-tools-core:
    initramfs-tools-core depends on klibc-utils (>= 2.0.4-8~); however:
    Package klibc-utils is not installed.
    initramfs-tools-core depends on cpio (>= 2.12); however:
    Package cpio is not installed.
    initramfs-tools-core depends on kmod; however:
    Package kmod is not installed.
    initramfs-tools-core depends on udev; however:
    Package udev is not installed.

dpkg: error processing package initramfs-tools-core (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
initramfs-tools-core

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants