Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support custom docker args for docker run command #277

Closed
solvingj opened this issue Oct 10, 2018 · 3 comments · Fixed by #343
Closed

Support custom docker args for docker run command #277

solvingj opened this issue Oct 10, 2018 · 3 comments · Fixed by #343

Comments

@solvingj
Copy link
Contributor

solvingj commented Oct 10, 2018

For example: CONAN_DOCKER_ARGS and an equivalent constructor argument.

We've discovered an issue in my company where the default settings for Windows Containers is to allocate 1GB of RAM. When building particularly large visual studio projects, we get a very random and intermittant compiler error. This occurs when using multiple versions of Visual Studio Build Tools 15.6.7, 15.7.6, and 15.8.6, and the error references a random C++ template parameter each time, from a variety of different libraries we use.

error C1001: An internal error has occurred in the compiler. 
compiler file 'msc1.cpp', line 1511

There are many reports about this error, and Microsoft explains that it's a very generic error that tells them almost nothing. For us, we've found that it doesn't happen unless we're in Docker, and only on one of our docker hosts, and only on this really big project. So, we discovered a workaround to be to pass the following additional argument to Docker:

-m 2gb

This doubles the RAM allocated to each container, and the problem seems to have gone away. However, it's very likely that simply applying this to all builds as part of CPT would have unexpected consequences for other users or even our other projects. Additionally, there are countless other docker flags which might be necessary to set under certain circumstances.

As a result, this request is for a primitive catch all string variable that would give users some control over the docker process they are launching, without complicating CPT too much.

@solvingj
Copy link
Contributor Author

solvingj commented Dec 2, 2018

There is some work being done right now to try to do Unreal Engine builds inside docker containers. Increasing the container memory is necessary for these builds:

https://adamrehn.com/articles/building-docker-images-for-unreal-engine-4/#issues-specific-to-windows-containers

So, would like to bump this if possible.

@uilianries
Copy link
Member

There are two ways to fix this, one is using your approach, adding a new generic argument where you could put anything but it's hard to debug and validate. Another option is the gitlab idea, where each option has a specific argument, easy to debug and validate, but very limited when we need to fix a new problem. I would prefer the first one, it's not safe but any new problem detected could be fixed immediately, without a new patch from CPT.

@solvingj
Copy link
Contributor Author

I think both is the best approach. Support the ones you know about with specific arguments, have the ability to add arbitrary for immediate fixes. Users are empowered to use both with whatever level of caution / discipline they choose.

@uilianries uilianries added this to the 0.25.0 milestone Mar 15, 2019
lasote added a commit that referenced this issue Mar 29, 2019
* allow passing user-specified Docker args during create step

* rename parameter

* mention CONAN_DOCKER_RUN_OPTIONS envvar in readme

* add tests

* #343 Improve docker integration test

- Fix conan_pip_package. It should be a str by default
- Pass original printer to be re-used by Docker runner
- Add integration test to validate docker run options

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #343 Fix skip test when running docker

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Add GitHub settings (#345) (#349)

* Add GitHub settings

- Use standard Conan tags for issues
- Add stale config for timeout bot
- Add issue template

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Use Github settings from conanio/docs

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #345 Add PR template

- Add PR template to generate the changelog
- Add the contributing guide

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Trigger Github Settings (#352)

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #336 Forward login method (#344)

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Upload package dependencies (#334)

* #237 Update README

- Add description about the new feature

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #237 Upload all dependencies

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #237 Unit test to upload all deps

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #237 Fix env var name

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #237 Fix upload dependencies

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #237 Upload only built packages

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #237 Only upload when package is available

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Accept custom pip command (#347)

* #147 Support custom PIP command

- Add env var CONAN_PIP_COMMAND which allows to run a specific
  pip version, or even a pip binary path.

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #147 Check pip command before to run

- To avoid command injection, the custom pip command will be checked

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #147 Test dynamic pip version

- Check for current pip version before to customize CONAN_PIP_COMMAND

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #147 Ignore which pip on Windows

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #147 pip install quiet mode

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #147 Try to fix Windows tests

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #277 Accept Docker arguments as list

- both string and list are accepted as docker arguments
- Update docker test to use list

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #277 Fix options when downloading docker image

- Docker is executed in two cases. The first one is when
  we need to download the docker image and patch with the
  newest Conan version. The options were skipped there

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #354 Add CONAN_CONANFILE option (#357)

* #354 Add CONAN_CONANFILE option

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #354 Add option description on README

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #354 Support CONAN_CONANFILE option

- Add full support for custom conanfile name
- Add tests

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #354 Fix boolean option

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #355 Skip check credentials is boolean (#356)

- Parse the env var CONAN_SKIP_CHECK_CREDENTIALS as boolean

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* fix azure pipelines branch detection (#346)

use variable BUILD_SOURCEBRANCH
BUILD_SOURCEBRANCHNAME only has the part after last /

* Feature/upload only recipe (#348)

* #153 Upload only the recipe

- Add CONAN_UPLOAD_ONLY_RECIPE env var, which set to skip
  all binary packages upload.

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #153 Test upload only the recipe

- get boolean value from environment variable is non-case sensitive now

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #153 Fix upload recipe argument

@madebr found a bug where the argument is wrong.

Co-Authored-By: uilianries <uilianries@gmail.com>

* #153 Fix unit test for upload recipe

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #153 Add test to check parameters

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #153 Fix conflict in README file

* Update Docker run example

- Use custom Docker network instead of using custom cache.

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Prepared for 1.15

* 0.25.0

* 1.15.0

* Fix Tox version (#364)

- Tox 3.8.1 is totally broken

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #361 Upload package with revisions (#363)

* #361 Upload package with revisions

- Package reference passed to be created is not the same
  retrieved from package id when the package is built.
  "foo/0.1@bar/testing" != "foo/0.1@bar/testing#<hash>"

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* #361 Parse revision

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Fix broken tests for release 0.25.0 (#365)

* Fix broken tests for release 0.25.0

- Release branch is fragile, only specific tests are able to run,
  or even CPT behavior could change during the tests.

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Run Docker test as regular test

- Do not require Bintray account to run docker tests

Signed-off-by: Uilian Ries <uilianries@gmail.com>

* Clean some stuff and improve test running in local
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants