Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build all options values #473

Merged
merged 9 commits into from
Mar 1, 2020
Merged

Build all options values #473

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
73fe6df
#457 Draft for build all options values
uilianries Feb 28, 2020
96fc642
#457 First try for Cartesian product
uilianries Feb 28, 2020
4029b1c
#457 Internalize matrix product in gens
uilianries Feb 29, 2020
500a0c4
#457 Update gcc generators test
uilianries Feb 29, 2020
61412d6
#457 Fix build tests
uilianries Feb 29, 2020
bfca52d
#457 Update build option on README
uilianries Feb 29, 2020
39d861d
#457 Update MSVC images on Appveyor [skip travis]
uilianries Feb 29, 2020
bf754b5
#457 Update MSVC images on Appveyor [skip travis]
uilianries Feb 29, 2020
ffe0a14
#457 Update generator tests
uilianries Feb 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ There are also two additional parameters of the ``add_common_builds``:
Pass "False" to deactivate it or "lib_name:shared_option_name" to specify a custom option name, e.j: boost:my_shared``
- **dll_with_static_runtime**: Will add also the combination of runtime MT with shared libraries.
- **header_only**: If your conanfile.py have an option **header_only**, the generated builds will contain automatically the "True/False" combination for that option [#454](https://github.com/conan-io/conan-package-tools/issues/454).
- **build_all_options_values**: It includes all possible values for the listed options [#457](https://github.com/conan-io/conan-package-tools/issues/457).

```
from cpt.packager import ConanMultiPackager
Expand Down Expand Up @@ -366,7 +367,19 @@ In case you want to integrate CPT with other tools, for example you want to have

Alternatively you can use the `CPT_SUMMARY_FILE` environment variable to set the summary file path

## Using all values for custom options
Sometimes you want to include more options to your matrix, including all possible combinations, so that, you can use **build_all_options_values**:

from cpt.packager import ConanMultiPackager


if __name__ == "__main__":
builder = ConanMultiPackager(reference="mypackage/0.1.0")
builder.add_common_builds(build_all_options_values=["mypackage:foo", "mypackage:bar"])
builder.run()

Now let's say mypackage's recipe contains the follow options: *shared*, *fPIC*, *foo* and *bar*. Both *foo* and *bar* can accept **True** or **False**.
The method add_common_builds will generate a matrix including both *foo* and *bar* with all possible combinations.

## Using Docker

Expand Down Expand Up @@ -1141,14 +1154,15 @@ The current commit message can contain special messages:

## Complete ConanMultiPackager methods reference:

- **add_common_builds(shared_option_name=None, pure_c=True, dll_with_static_runtime=False, reference=None, header_only=True)**: Generate a set of package configurations and add them to the
- **add_common_builds(shared_option_name=None, pure_c=True, dll_with_static_runtime=False, reference=None, header_only=True, build_all_options_values=None)**: Generate a set of package configurations and add them to the
list of packages that will be created.

- **shared_option_name**: If given, ConanMultiPackager will add different configurations for -o shared=True and -o shared=False.
- **pure_c**: ConanMultiPackager won't generate different builds for the **libstdc++** c++ standard library, because it is a pure C library.
- **dll_with_static_runtime**: generate also build for "MT" runtime when the library is shared.
- **reference**: Custom package reference
- **header_only**: Generate new builds following header-only options [#454](https://github.com/conan-io/conan-package-tools/issues/454)
- **build_all_options_values**: Include all values for the listed options [#457](https://github.com/conan-io/conan-package-tools/issues/457)

- **login(remote_name)**: Performs a `conan user` command in the specified remote.

Expand Down Expand Up @@ -1209,6 +1223,7 @@ This is especially useful for CI integration.
- **CONAN_ARCHS**: Architectures to build for, comma separated, e.g. "x86,x86_64"
- **CONAN_OPTIONS**: Conan build options, comma separated, e.g. "foobar:with_bar=True,foobar:with_qux=False"
- **CONAN_SHARED_OPTION_NAME**: Set `shared_option_name` by environment variable, e.g. "mypackagename:shared"
- **CONAN_BUILD_ALL_OPTIONS_VALUES**: Set `build_all_options_values` by environment variable, e.g. "mypackagename:foo,mypackagename:bar"
- **CONAN_BUILD_TYPES**: Build types to build for, comma separated, e.g. "Release,Debug"
- **CONAN_CPPSTDS**: List containing values for `compiler.cppstd`. Default None
- **CONAN_VISUAL_VERSIONS**: Visual versions, comma separated, e.g. "12,14"
Expand Down
113 changes: 89 additions & 24 deletions cpt/builds_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
from conans.model.version import Version
from cpt.tools import split_colon_env, transform_list_options_to_dict

default_gcc_versions = ["4.9", "5", "6", "7", "8"]
default_clang_versions = ["3.8", "3.9", "4.0", "5.0", "6.0", "7.0"]
default_gcc_versions = ["4.9", "5", "6", "7", "8", "9"]
default_clang_versions = ["3.8", "3.9", "4.0", "5.0", "6.0", "7.0", "8", "9"]
default_visual_versions = ["14", "15"]
default_visual_runtimes = ["MT", "MD", "MTd", "MDd"]
default_apple_clang_versions = ["9.0", "9.1", "10.0"]
default_apple_clang_versions = ["9.0", "9.1", "10.0", "11.0"]
default_archs = ["x86", "x86_64"]
default_build_types = ["Release", "Debug"]
default_cppstds = [None]
Expand Down Expand Up @@ -142,37 +142,44 @@ def __init__(self, reference, os_name, gcc_versions, apple_clang_versions, clang
options = []
self._options = transform_list_options_to_dict(options)

def get_builds(self, pure_c, shared_option_name, dll_with_static_runtime, reference=None):
def get_builds(self, pure_c, shared_option_name, dll_with_static_runtime, reference=None,
build_all_options_values=None):

ref = reference or self._reference

if self._os_name == "Windows":
if self._mingw_configurations:
builds = get_mingw_builds(self._mingw_configurations,
get_mingw_package_reference(), self._archs,
shared_option_name, self._build_types, self._cppstds, self._options, ref)
shared_option_name, self._build_types, self._cppstds,
self._options, ref, build_all_options_values)
else:
builds = []
builds.extend(get_visual_builds(self._visual_versions, self._archs,
self._visual_runtimes, self._visual_toolsets,
shared_option_name, dll_with_static_runtime,
self._vs10_x86_64_enabled,
self._build_types, self._cppstds, self._options, ref))
self._build_types, self._cppstds, self._options, ref,
build_all_options_values))
return builds
elif self._os_name == "Linux":
builds = get_linux_gcc_builds(self._gcc_versions, self._archs, shared_option_name,
pure_c, self._build_types, self._cppstds, self._options, ref)
pure_c, self._build_types, self._cppstds, self._options, ref,
build_all_options_values)
builds.extend(get_linux_clang_builds(self._clang_versions, self._archs,
shared_option_name, pure_c, self._build_types,
self._cppstds, self._options, ref))
self._cppstds, self._options, ref,
build_all_options_values))
return builds
elif self._os_name == "Darwin":
return get_osx_apple_clang_builds(self._apple_clang_versions, self._archs,
shared_option_name, pure_c, self._build_types,
self._cppstds, self._options, ref)
self._cppstds, self._options, ref,
build_all_options_values)
elif self._os_name == "FreeBSD":
return get_linux_clang_builds(self._clang_versions, self._archs, shared_option_name,
pure_c, self._build_types, self._cppstds, self._options, ref)
pure_c, self._build_types, self._cppstds, self._options,
ref, build_all_options_values)
else:
raise Exception("Unknown operating system: %s" % self._os_name)

Expand All @@ -195,13 +202,13 @@ def __new__(cls, settings, options, env_vars, build_requires, reference):
if isinstance(reference, str):
reference = ConanFileReference.loads(reference)

return super(BuildConf, cls).__new__(cls, settings, options, env_vars,
build_requires, reference)
return super(BuildConf, cls).__new__(cls, settings, options, env_vars, build_requires,
reference)


def get_mingw_builds(mingw_configurations, mingw_installer_reference,
archs, shared_option_name, build_types, cppstds,
options, reference=None):
options, reference=None, build_all_options_values=None):
builds = []
for config in mingw_configurations:
version, arch, exception, thread = config
Expand All @@ -213,11 +220,16 @@ def get_mingw_builds(mingw_configurations, mingw_installer_reference,
"compiler.exception": exception}
build_requires = {"*": [mingw_installer_reference]}

if shared_option_name:
if shared_option_name and not build_all_options_values:
for shared in [True, False]:
opt = copy.copy(options)
opt[shared_option_name] = shared
builds += _make_mingw_builds(settings, opt, build_requires, build_types, cppstds, reference)
elif build_all_options_values:
for option_values in build_all_options_values:
opt = copy.copy(options)
opt.update(option_values)
builds += _make_mingw_builds(settings, opt, build_requires, build_types, cppstds, reference)
else:
builds += _make_mingw_builds(settings, copy.copy(options), build_requires, build_types, cppstds, reference)

Expand All @@ -241,7 +253,7 @@ def _make_mingw_builds(settings, options, build_requires, build_types, cppstds,

def get_visual_builds(visual_versions, archs, visual_runtimes, visual_toolsets, shared_option_name,
dll_with_static_runtime, vs10_x86_64_enabled, build_types, cppstds,
options, reference=None):
options, reference=None, build_all_options_values=None):

visual_toolsets = visual_toolsets or get_env_visual_toolsets()
ret = []
Expand All @@ -258,14 +270,16 @@ def get_visual_builds(visual_versions, archs, visual_runtimes, visual_toolsets,
visual_builds = get_visual_builds_for_version(visual_runtimes, visual_version, arch,
shared_option_name,
dll_with_static_runtime, build_types,
cppstds, options, reference, toolset=toolset)
cppstds, options, reference,
toolset=toolset,
build_all_options_values=build_all_options_values)
ret.extend(visual_builds)
return ret


def get_visual_builds_for_version(visual_runtimes, visual_version, arch, shared_option_name,
dll_with_static_runtime, build_types, cppstds, options,
reference=None, toolset=None):
reference=None, toolset=None, build_all_options_values=None):
base_set = {"compiler": "Visual Studio",
"compiler.version": visual_version,
"arch": arch}
Expand Down Expand Up @@ -294,7 +308,7 @@ def get_visual_builds_for_version(visual_runtimes, visual_version, arch, shared_
partial_settings = {"build_type": bld, "compiler.runtime": rt}
if cppstd:
partial_settings["compiler.cppstd"] = cppstd
if shared_option_name:
if shared_option_name and not build_all_options_values:
opt = copy.copy(options)
opt[shared_option_name] = False
sets.append((partial_settings, opt, {}, {}))
Expand All @@ -309,6 +323,15 @@ def get_visual_builds_for_version(visual_runtimes, visual_version, arch, shared_
opt[shared_option_name] = True
sets.append((partial_settings,
opt, {}, {}))
elif build_all_options_values:
for option_values in build_all_options_values:
opt = copy.copy(options)
opt.update(option_values)
sets.append((partial_settings, opt, {}, {}))
if shared_option_name and rt in ['MT', 'MTd'] and dll_with_static_runtime:
new_opt = copy.copy(opt)
new_opt[shared_option_name] = True
sets.append((partial_settings, opt, {}, {}))
else:
sets.append((partial_settings, options, {}, {}))

Expand Down Expand Up @@ -336,13 +359,14 @@ def get_build(compiler, the_arch, the_build_type, the_compiler_version,


def get_osx_apple_clang_builds(apple_clang_versions, archs, shared_option_name,
pure_c, build_types, cppstds, options, reference=None):
pure_c, build_types, cppstds, options, reference=None,
build_all_options_values=None):
ret = []
# Not specified compiler or compiler version, will use the auto detected
for compiler_version in apple_clang_versions:
for arch in archs:
for cppstd in cppstds:
if shared_option_name:
if shared_option_name and not build_all_options_values:
for shared in [True, False]:
opt = copy.copy(options)
opt[shared_option_name] = shared
Expand All @@ -355,6 +379,19 @@ def get_osx_apple_clang_builds(apple_clang_versions, archs, shared_option_name,
ret.append(get_build("apple-clang", arch, build_type_it,
compiler_version, None,
None, opt, reference))
elif build_all_options_values:
for option_values in build_all_options_values:
opt = copy.copy(options)
opt.update(option_values)
for build_type_it in build_types:
if not pure_c:
ret.append(get_build("apple-clang", arch, build_type_it,
compiler_version, cppstd,
"libc++", opt, reference))
else:
ret.append(get_build("apple-clang", arch, build_type_it,
compiler_version, None,
None, opt, reference))
else:
for build_type_it in build_types:
if not pure_c:
Expand All @@ -370,13 +407,13 @@ def get_osx_apple_clang_builds(apple_clang_versions, archs, shared_option_name,


def get_linux_gcc_builds(gcc_versions, archs, shared_option_name, pure_c, build_types, cppstds,
options, reference=None):
options, reference=None, build_all_options_values=None):
ret = []
# Not specified compiler or compiler version, will use the auto detected
for gcc_version in gcc_versions:
for arch in archs:
for cppstd in cppstds:
if shared_option_name:
if shared_option_name and not build_all_options_values:
for shared in [True, False]:
opt = copy.copy(options)
opt[shared_option_name] = shared
Expand All @@ -390,6 +427,21 @@ def get_linux_gcc_builds(gcc_versions, archs, shared_option_name, pure_c, build_
else:
ret.append(get_build("gcc", arch, build_type_it, gcc_version,
None, None, opt, reference))
elif build_all_options_values:
for option_values in build_all_options_values:
opt = copy.copy(options)
opt.update(option_values)
for build_type_it in build_types:
if not pure_c:
ret.append(get_build("gcc", arch, build_type_it, gcc_version,
cppstd, "libstdc++", opt, reference))
if float(gcc_version) >= 5:
ret.append(get_build("gcc", arch, build_type_it, gcc_version,
cppstd, "libstdc++11", opt, reference))
else:
ret.append(get_build("gcc", arch, build_type_it, gcc_version,
None, None, opt, reference))

else:
for build_type_it in build_types:
if not pure_c:
Expand All @@ -405,13 +457,13 @@ def get_linux_gcc_builds(gcc_versions, archs, shared_option_name, pure_c, build_


def get_linux_clang_builds(clang_versions, archs, shared_option_name, pure_c, build_types, cppstds,
options, reference=None):
options, reference=None, build_all_options_values=None):
ret = []
# Not specified compiler or compiler version, will use the auto detected
for clang_version in clang_versions:
for arch in archs:
for cppstd in cppstds:
if shared_option_name:
if shared_option_name and not build_all_options_values:
for shared in [True, False]:
opt = copy.copy(options)
opt[shared_option_name] = shared
Expand All @@ -424,6 +476,19 @@ def get_linux_clang_builds(clang_versions, archs, shared_option_name, pure_c, bu
else:
ret.append(get_build("clang", arch, build_type_it, clang_version,
None, None, opt, reference))
elif build_all_options_values:
for option_values in build_all_options_values:
opt = copy.copy(options)
opt.update(option_values)
for build_type_it in build_types:
if not pure_c:
ret.append(get_build("clang", arch, build_type_it, clang_version,
cppstd, "libstdc++", opt, reference))
ret.append(get_build("clang", arch, build_type_it, clang_version,
cppstd, "libc++", opt, reference))
else:
ret.append(get_build("clang", arch, build_type_it, clang_version,
None, None, opt, reference))
else:
for build_type_it in build_types:
if not pure_c:
Expand Down
Loading