Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a --prefer-binary flag to 'pip install' command #2096

Merged
merged 2 commits into from
Jan 24, 2023

Conversation

tenzen-y
Copy link
Member

@tenzen-y tenzen-y commented Jan 24, 2023

Signed-off-by: Yuki Iwai yuki.iwai.tz@gmail.com

What this PR does / why we need it:
I pinned the h5py version with 3.7.0 to resolve image-building errors. These errors are caused by the latest h5py not releasing the wheel for aarch64.

I added a --prefer-binary flag to pip install command to resolve the following build errors.

Doc: https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-prefer-binary

Also, these errors are caused by the latest h5py not releasing the binary for aarch64.

...
 > [linux/arm64 stage-1 7/8] RUN pip install --no-cache-dir -r requirements.txt:
#27 136.0       running build_ext
#27 136.0       Building h5py requires pkg-config unless the HDF5 path is explicitly specified using the environment variable HDF5_DIR. For more information and details, see https://docs.h5py.org/en/stable/build.html#custom-installation
#27 136.0       error: pkg-config probably not installed: FileNotFoundError(2, 'No such file or directory')
#27 136.0       [end of output]
#27 136.0   
#27 136.0   note: This error originates from a subprocess, and is likely not a problem with pip.
#27 136.0 Successfully built googleapis-common-protos
#27 136.0   ERROR: Failed building wheel for h5py
#27 136.0 Failed to build h5py
#27 136.0 ERROR: Could not build wheels for h5py, which is required to install pyproject.toml-based projects

Ref: h5py/h5py#2215

Blocking #2077 #2091 #2092 #2094

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Checklist:

  • Docs included if any changes are user facing

Signed-off-by: Yuki Iwai <yuki.iwai.tz@gmail.com>
@@ -13,7 +13,7 @@ ENV TARGET_DIR /opt/katib
ENV SUGGESTION_DIR cmd/suggestion/hyperopt/v1beta1
ENV PYTHONPATH ${TARGET_DIR}:${TARGET_DIR}/pkg/apis/manager/v1beta1/python:${TARGET_DIR}/pkg/apis/manager/health/python

RUN if [ "${TARGETARCH}" = "ppc64le" ] || [ "${TARGETARCH}" = "aarch64" ]; then \
RUN if [ "${TARGETARCH}" = "ppc64le" ]; then \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BuildKit propagates TARGETARCH with amd64 or arm64. However, we have been able to build container images for the platform linux/arm64 with an incorrect value, aarch64.

This means we don't need these packages for the platform linux/arm64.

@tenzen-y
Copy link
Member Author

I found a better solution.
/hold

@tenzen-y tenzen-y changed the title Pin the h5py version with 3.7.0 [WIP] Pin the h5py version with 3.7.0 Jan 24, 2023
Signed-off-by: Yuki Iwai <yuki.iwai.tz@gmail.com>
@google-oss-prow google-oss-prow bot added size/M and removed size/S labels Jan 24, 2023
@tenzen-y tenzen-y changed the title [WIP] Pin the h5py version with 3.7.0 Add a --prefer-binary flag to 'pip install' command Jan 24, 2023
@tenzen-y
Copy link
Member Author

/hold cancel

@johnugeorge
Copy link
Member

LGTM
/assign @andreyvelich

Copy link
Member

@andreyvelich andreyvelich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the prompt fix @tenzen-y !
/lgtm
/approve

@google-oss-prow google-oss-prow bot added the lgtm label Jan 24, 2023
@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich, tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [andreyvelich,tenzen-y]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit a44aaea into kubeflow:master Jan 24, 2023
@tenzen-y tenzen-y deleted the fix-image-build-errors branch January 24, 2023 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants