Skip to content
This repository has been archived by the owner on Aug 2, 2024. It is now read-only.

feat: Improve docker-compose performance #483

Conversation

d-roak
Copy link
Collaborator

@d-roak d-roak commented May 29, 2023

This PR contains some small improvements in the docker-compose performance.

Pull Request type

Feature

Please add the labels corresponding to the type of changes your PR introduces:

  • Feature
  • Build-related changes

What is the current behavior?

  • There are warnings while running the docker-compose when not running it in amd64 architectures (tested in a Mac M1) due to using paritytech/ci-linux as the base container
  • When you run amd64 containers on Mac, they are runned with some emulator, which leads Cargo to use way more memory (users.rust-lang.org, rust-lang/cargo#10781). In some systems, the cargo build --release will crash while updating the crates.io index

Resolves: #NA

What is the new behavior?

  • Specified the platform for the container, this way there are no more warnings. Although the performance is not improved by doing this
  • Use Cargo's new sparse index protocol to reduce the memory footprint of updating the crates.io index (Cargo Sparse Protocol), leading to some performance improvements
    • The sparse protocol seems to be a nice approach to be used not only for the docker build, but also for the whole project config
    • An alternative that also reduces the memory footprint (I tested, and it works), is to use the command-line git instead of libgit2. We can do that by appending --config net.git-fetch-with-cli=true to the build command, or add that configurations in the Cargo config file

Does this introduce a breaking change?

No

Other information

This PR was opened as a draft for us to discuss if this is the best approach to go and ignore performance issues on aarch64 (such as Macs Mx and RPi), or if we should support docker on other architectures (replacing paritytech/ci-linux base image).

@codecov
Copy link

codecov bot commented May 29, 2023

Codecov Report

Patch coverage has no change and project coverage change: +0.45 🎉

Comparison is base (d7c6119) 34.16% compared to head (d415820) 34.61%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #483      +/-   ##
==========================================
+ Coverage   34.16%   34.61%   +0.45%     
==========================================
  Files          63       63              
  Lines        6960     6930      -30     
  Branches     6960     6930      -30     
==========================================
+ Hits         2378     2399      +21     
+ Misses       3822     3763      -59     
- Partials      760      768       +8     

see 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@AbdelStark AbdelStark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@dbejarano820
Copy link
Contributor

I'm excited for this one, docker-compose has been crashing on my Mac M1

@d-roak
Copy link
Collaborator Author

d-roak commented May 29, 2023

Do you think we should create a custom base image to support aarch64?

@0xLucqs 0xLucqs added feature New feature devops labels May 30, 2023
@0xLucqs 0xLucqs force-pushed the feature/improve-docker-performance branch from 20bfefb to 7edc8c3 Compare May 30, 2023 11:58
@0xLucqs 0xLucqs force-pushed the feature/improve-docker-performance branch from 7edc8c3 to 3e907b2 Compare May 31, 2023 14:56
@d-roak d-roak changed the title feat: Improve docker-compose performance WIP feat: Improve docker-compose performance May 31, 2023
@d-roak
Copy link
Collaborator Author

d-roak commented Jun 1, 2023

I'll have to test some docker deploys with gh actions in my fork to refactor the docker files. In the meantime, we can merge this one as a temporary fix for running docker in Mac M1

@dbejarano820 after this is merged, please confirm that you can run the docker-compose commands on your Mac

@d-roak d-roak changed the title WIP feat: Improve docker-compose performance feat: Improve docker-compose performance Jun 1, 2023
@d-roak d-roak marked this pull request as ready for review June 1, 2023 12:27
@AbdelStark AbdelStark merged commit 16fe981 into keep-starknet-strange:main Jun 2, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Jun 4, 2023
@d-roak d-roak deleted the feature/improve-docker-performance branch June 5, 2023 18:19
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants