Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking: remove harmful use of export in make and immediate evaluation #10850

Closed
19 tasks done
cladmi opened this issue Jan 23, 2019 · 4 comments
Closed
19 tasks done

Tracking: remove harmful use of export in make and immediate evaluation #10850

cladmi opened this issue Jan 23, 2019 · 4 comments
Labels
Area: build system Area: Build system Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Type: tracking The issue tracks and organizes the sub-tasks of a larger effort

Comments

@cladmi
Copy link
Contributor

cladmi commented Jan 23, 2019

Description

Many variables are exported using export VARIABLE without even knowing why, the impact, or even be needed. In the same time, many variables are evaluated using := instead of using deferred variables.

Guess why calling make clean is slow ? All these variables/shell call must be evaluated for each target called even if the targett is not using it at all.

I tried preventing changing these without reasons, but keeping them in this wrong state by default, is causing me issues in several places. I want to start getting rid of them on a methodical way instead of currently working it around.

Listing them all

git grep -a -n 'export ' '*' ':!doc/**' ':!**/*.c' ':!**/*.h' ':!**/*.py' ':!makefiles/utils/**' ':!*.murdock' ':!dist/tools/buildsystem_sanity_check/**' ':!*release-notes.txt' ':!dist/tools/**' | tee output
wc -l output; date
1105
Fri May 17 13:37:10 CEST 2019
wc -l output ; date
938 output
Tue May 28 18:23:42 CEST 2019
wc -l output ; date  # Pre rule update
940 output
Mon Jun  3 10:01:49 CEST 2019
wc -l output ; date  # Grep rule updated
892 output
Mon Jun  3 10:04:58 CEST 2019
wc -l output; date
854 output
Mon Jun  3 10:46:56 CEST 2019
wc -l output; date
830 output
Tue Jun  4 16:04:14 CEST 2019
wc -l output; date
822 output
Tue Jun 11 13:31:51 CEST 2019
wc -l output; date
619 output
Wed Aug 21 20:17:48 CEST 2019
wc -l output; date  # rule update to see in patches and match 'export '
618 output
Thu Aug 29 11:51:28 CEST 2019
wc -l output; date
498 output
Thu Aug 29 14:39:32 CEST 2019

wc -l output ; date
503 output
Thu Sep 12 14:47:10 CEST 2019
wc -l output; date
280 output
Wed Jun 24 08:32:23 CEST 2020

I am personally tracking them in https://ci-ilab.imp.fu-berlin.de/job/RIOT%20cleanup%20warnings/

Steps

Cleaning up

This has different consequences:

Evaluation when not needed
  • Using an export on a variable means the value will be evaluated even for rules that do not use the variable.
    This causes issues for example with listing tty devices, evaluating INCLUDES and getting GCC_INCLUDES_DIRS

  • It also causes evaluation on the host machine when only the docker container should be doing it. It happened with avr-ld for example, some checks being performed on the host machine, saying objcopy not found.

Sub builds

When trying to call make inside of make many variables from the first build are sent to the second one.

Variables that can change a build

As everything is exported, it is hard to know what can have an impact on a build result.
Having it in control would simplify documenting what should and what should not be available.

The default behavior has become harmful

The default case has been to always use := or export even when it does not make sense, and it gets merged without even reviewing why there is an export there.

export PRIVATE_VAR_FOR_SCRIPT = foo_bar
OTHER_VAR_ONLY_USE_OF_PRIVATE_VAR_FOR_SCRIPT = $(PRIVATE_VAR_FOR_SCRIPT)

Examples

Related issues

I am starting to build on a machine that has nothing but docker and the build system complains about these things.

@cladmi cladmi added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Area: build system Area: Build system Type: tracking The issue tracks and organizes the sub-tasks of a larger effort labels Jan 23, 2019
@cladmi
Copy link
Contributor Author

cladmi commented Feb 25, 2019

Exporting TERMPROG and TERMFLAGS for nothing breaks setting RIOT_TERMINAL from within testrunner as tried in #10952. And such for no reason !

@cladmi cladmi changed the title Tracking: remove harmfull use of export in make and immediate evaluation Tracking: remove harmful use of export in make and immediate evaluation Jun 7, 2019
@aabadie
Copy link
Contributor

aabadie commented Jun 24, 2020

@fjmolinas do you think issue could be closed ? It seems all points were addressed.

@fjmolinas
Copy link
Contributor

Not sure, moving the to vars.inc.mk is a first step, but maybe some of them do not need exporting at all?\

@maribu
Copy link
Member

maribu commented May 24, 2023

Let's see if someone complains when I close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: build system Area: Build system Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Type: tracking The issue tracks and organizes the sub-tasks of a larger effort
Projects
None yet
Development

No branches or pull requests

5 participants