Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Dockerfile #12

Merged
merged 20 commits into from
Nov 30, 2024
Merged

Create Dockerfile #12

merged 20 commits into from
Nov 30, 2024

Conversation

bfrizzell01
Copy link
Collaborator

  • added Python 3.11 dependency to environment.yml
  • removed conda-lock.yml, replaced with conda-linux-64.lock
  • added Dockerfile
  • added docker publish action

@bfrizzell01
Copy link
Collaborator Author

Todo: Add deepcheck package to conda-lock

@bfrizzell01
Copy link
Collaborator Author

Also add brief instructions for running docker container

Copy link
Collaborator

@MasonZhang-MZ MasonZhang-MZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for setting up the environment, Ben. Just spotted 2 small issues:

  1. deepchecks version confirmation in environment.yml
  2. Automate the installation for deepchecks in Dockerfile

Please see comments on environment.yml and Dockerfile for detail.

RUN mamba update --quiet --file /tmp/conda-linux-64.lock \
&& mamba clean --all -y -f \
&& fix-permissions "${CONDA_DIR}" \
&& fix-permissions "/home/${NB_USER}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might need to automate the installation for deepchecks during the build process by adding RUN pip install deepchecks==0.3.2 at the end of the Dockerfile (for deepchecks version, please refer to the comment on the environment.yml file to confirm which version to install).

Copy link
Collaborator Author

@bfrizzell01 bfrizzell01 Nov 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the deepchecks package not included on your end when you run the container? It seems to properly install on my end. Let me know, and if you have issues I will make sure to implement this.

Update: refer to my following comment about the environment.yml file regarding this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

container runs fine on my end with deepchecks installed, thanks @bfrizzell01

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the requested line to Dockerfile. Data Validation should now work properrly, @y1chi-z .

- python=3.11
- pandera=0.21.0
- deepchecks=0.3.2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just double-checking for the pinned version of deepchecks, any particular reason of using 0.3.2? 0.18.1 should be the latest release, if there are any compatibility issue with this version, please disregard this comment.

Copy link
Collaborator Author

@bfrizzell01 bfrizzell01 Nov 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.3.2 was the version that conda automatically installed into our environment with the other packages, so i'm assuming that is the version that is most compatible. But I can try it with the newer version and see if there are any issues.

Update: It does seem that version 0.18.1 has conflicts with other pinned package versions, so I've added a line to Dockerfile to include the newest version.

@MasonZhang-MZ MasonZhang-MZ linked an issue Nov 30, 2024 that may be closed by this pull request
@MasonZhang-MZ MasonZhang-MZ merged commit a7fb40c into main Nov 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Potential Improvement for environment.yml and Dockerfile
3 participants