[bug] In build-order.json, build_args is wrong when options have whitespaces

[jeremy-coulon]

Describe the bug

conan version: 2.4.1
conan profile:

[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=12
os=Linux

Description:

I am trying to build boost/1.82.0 package in CI with some custom options and in particular: boost/*:extra_b2_flags="define=BOOST_LOG_USE_STD_REGEX define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_DEFAULT_FACTORIES define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_IPC define=BOOST_LOG_WITHOUT_SETTINGS_PARSERS define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_WCHAR_T define=BOOST_USE_WINAPI_VERSION=0x0A00"
This option contains whitespaces.
When using conan graph build-order --format json, the resulting json has a build_args field that looks like: --requires=boost/1.82.0 --build=boost/1.82.0 -o boost/*:extra_b2_flags=define=BOOST_LOG_USE_STD_REGEX define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_DEFAULT_FACTORIES define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_IPC define=BOOST_LOG_WITHOUT_SETTINGS_PARSERS define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_WCHAR_T define=BOOST_USE_WINAPI_VERSION=0x0A00
This is not valid arguments for conan install because extra_b2_flags option should be enclosed in quotes.

How to reproduce it

With the following conanfile.txt:

[requires]
boost/1.82.0

[options]
boost/*:extra_b2_flags=define=BOOST_LOG_USE_STD_REGEX define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT define=BOOST_LOG_WITHOUT_DEFAULT_FACTORIES define=BOOST_LOG_WITHOUT_EVENT_LOG define=BOOST_LOG_WITHOUT_IPC define=BOOST_LOG_WITHOUT_SETTINGS_PARSERS define=BOOST_LOG_WITHOUT_SYSLOG define=BOOST_LOG_WITHOUT_WCHAR_T define=BOOST_USE_WINAPI_VERSION=0x0A00

Run the following:
$ conan graph build-order --order-by=configuration --build=boost/1.82.0 -pr:a default --format json .

Inspect the resulting json and look at build_args field which is invalid.

[AbrilRBS]

Hi @jeremy-coulon thanks a lot for taking the time to report the issue.
While it is true that Conan silently spliting the args is not quite expected without even a warning.

If I change
boost/*:extra_b2_flags=define=BOOST_LOG_USE_STD_REGEX ... define=BOOST_USE_WINAPI_VERSION=0x0A00
with
boost/*:extra_b2_flags="define=BOOST_LOG_USE_STD_REGEX ... define=BOOST_USE_WINAPI_VERSION=0x0A00",
the json output correctly gets converted to

boost/*:extra_b2_flags=\"define=BOOST_LOG_USE_STD_REGEX define=BOOST_LOG_WITHOUT_DEBUG_OUTPUT ... define=BOOST_USE_WINAPI_VERSION=0x0A00\"

But this probably won't work within the boost recipe, as those quotes are now being passed all the way down to the compiler, which will probably complain about them when you try to create the package.

A proper solution here would be to have boost/b2 listen to the confs that conan provides for this exact reason, such as tools.build:defines etc. I'm talking about how we can best proceed there with @uilianries.

A question thus remains as to how Conan could help users in this case - we could try to quote them when necessary, we'll check and get back to you on that :)

[AbrilRBS]

I'm porposing #16594 which will quote the arguments in the build_args field to avoid issues like this. This might be just in time for the 2.5.0 release, will check with the team.

Again, thanks for taking the time to report your issue, we appreciate it!

[jeremy-coulon]

Thanks for the pull request.

As for your suggestion to add quotes directly in my conanfile.txt, note that I already tried this workaround.
But it turned out to have an even worse problem. Now the package_id computed by conan graph build-order is different than the package_id computed by conan install with the build_args taken from the result of the build-order.

[AbrilRBS]

Now the package_id computed by conan graph build-order is different than the package_id computed by conan install with the build_args taken from the result of the build-order.

This is because the quotes would be part of the options in one case and not in another, which messes up things. I think we need to rethink how we either parse or output the information to users, so this might take a bit, will keep you posted :)

[nextsilicon-itay-bookstein]

I just took a shot at writing some parallelization automation around conan graph build-order and wondered about the same thing. I think it's worth:

1. Documenting what the escape syntax is (or will be), so that automations can be robust in relation to this. For instance, is shlex.split expected to work correctly?
2. Maybe also offering an alternative in the form of a JSON list?

Thanks!

[memsharded]

#16594 closed this, proposing double quotes to the arguments.
Please check it and if it is still not enough for your case, create a new ticket with the details, and we will try to add a full json format of the args, so users can compose the command line specific to their needs.

Thanks for your feedback!