-Werror=declaration-after-statement is added even for extension modules through setup.py

[nilsge]

BPO	21121
Nosy	@ronaldoussoren, @pitrou, @larryhastings, @benjaminp, @ned-deily, @merwok, @skrah, @dstufft, @wm75
Files	pybug.txt: setup.py and also the error log issue21121.diff issue21121-2.diff: improved version of issue21121.diff issue21121-3.diff: alternative approach

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-08-10.03:04:30.723>
created_at = <Date 2014-04-01.10:42:13.877>
labels = ['extension-modules', 'build', 'library']
title = '-Werror=declaration-after-statement is added even for extension modules through setup.py'
updated_at = <Date 2014-08-10.03:04:30.712>
user = 'https://bugs.python.org/nilsge'

bugs.python.org fields:

activity = <Date 2014-08-10.03:04:30.712>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2014-08-10.03:04:30.723>
closer = 'python-dev'
components = ['Build', 'Distutils', 'Extension Modules']
creation = <Date 2014-04-01.10:42:13.877>
creator = 'nilsge'
dependencies = []
files = ['34692', '34909', '35133', '35134']
hgrepos = []
issue_num = 21121
keywords = ['patch']
message_count = 22.0
messages = ['215305', '215348', '215350', '216467', '216476', '216879', '217781', '217805', '217842', '217847', '217865', '217869', '217870', '217881', '217882', '217888', '217891', '217915', '218000', '218001', '224736', '225122']
nosy_count = 12.0
nosy_names = ['ronaldoussoren', 'pitrou', 'larry', 'benjamin.peterson', 'ned.deily', 'eric.araujo', 'Arfrever', 'skrah', 'python-dev', 'dstufft', 'wolma', 'nilsge']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue21121'
versions = ['Python 3.4', 'Python 3.5']

[nilsge]

I got an error while rebuilding a module for 3.4. This was a ISO C90 error but setup.py explicitely adds "-std=c99" to the gcc parameters, and indeed it is used.

fifo.h:114:5: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
uint32_t ofs = fifo->write_count - fifo->write_offset;

However, Py 3.4 seems to add -Werror=declaration-after-statement also for extension modules. This should not happe (said also Yhg1s in #python).

Attached is a file that shows the setup.py and also the error log.

[ned-deily]

It looks like -Werror=declaration-after-statement was added to BASECFLAGS in configure.ac during the 3.4 development cycle by a3559c8c614b and e47806951fb2. Unfortunately, BASECFLAGS propagates through to extension module builds as well. If -Werror=declaration-after-statement should only be restricted to the build of the interpreter executable itself, one option *might be* to move the test and definition to CFLAGSFORSHARED in configure.ac.

A workaround could be to define CFLAGS before rebuilding a module:

export CFLAGS=$(python3.4 -c 'import sysconfig; print(sysconfig.get_config_var("CFLAGS").replace("-Werror=declaration-after-statement",""))')

(As usual, my brain hurts after trying to sift through the myriad build flags and their interactions in configure.ac, Makefile, and setup.py.)

[nilsge]

The workaround indeed works. At least I can work now until this gets officialy fixed. Thanks!

export CFLAGS=$(python3.4 -c 'import sysconfig; print(sysconfig.get_config_var("CFLAGS").replace("-Werror=declaration-after-statement",""))')

[skrah]

Here is a patch. I do not see a really nice way to deal with the problem.
The cleanest way I found was to introduce a new Makefile variable CFLAGS_NODIST
and use that in the interpreter and stdlib build.

[wm75]

I ran into this issue right after 3.4 got released.

I solved it by adding

extra_compile_args=["-Wno-error=declaration-after-statement"]

as an argument to the Extension() call in the package's setup.py .

[ned-deily]

Stefan, the patch LGTM although I sure wish we were removing some CFLAGS-related configuration variables rather than adding another. But I don't have a better suggestion short of a comprehensive cleanup of all of them.

[skrah]

Thanks, Ned. I'm attaching a second version of the existing patch
with improved error handling and a fix for test_distutils, which
failed.

The result is slightly overcomplicated, so I came up with a
different approach in bpo-21121-3.diff. Thoughts?

[merwok]

I like bpo-21121-3.diff.

[ned-deily]

I agree: bpo-21121-3.diff is a much better approach.

[larryhastings]

If you guys want this in 3.4.1, please get it checked in in the next, oh, eight hours.

[skrah]

If you guys want this in 3.4.1, please get it checked in in the next, oh, eight hours.

I can't commit today. Perhaps one of you wants to take over (I think we all
agree that the third patch is the best).

[larryhastings]

Sorry, I should have said "3.4.1rc1". You can still get it in for 3.4.1.

[ronaldoussoren]

This is the same issue as bpo-18211. As that issue doesn't have a patch and this one does, I'm closing bpo-18211 as a duplicate.

[merwok]

I can commit the patch but won’t be able to check the buildbots for the next twelve hours.

[ned-deily]

There's no immediate rush now. It's too late for 3.4.1rc1.

[skrah]

One more question:

I think it's nicer to add CFLAGS_NODIST to 'renamed_variables' in
Lib/sysconfig.py:265:

    renamed_variables = ('CFLAGS', 'CFLAGS_NODIST', 'LDFLAGS', 'CPPFLAGS')

That way it's possible to look up CFLAGS_NODIST directly.

For consistency, we can do the same for Lib/distutils/sysconfig.py,
where the variable would be purely informational.

[ned-deily]

Is there any reason to expose CFLAGS_NODIST externally? It seems to me that it is only needed in the top-level setup.py for building standard library extension modules. Let's not add yet another configuration variable to the already confusing array we present to users through the two sysconfig.get_config_var().

[skrah]

The current patch allows the user to specify e.g.:

CFLAGS_NODIST="-march=core2" ./configure

So it would be surprising to get:

>>> import sysconfig
>>> sysconfig.get_config_var('CFLAGS_NODIST')
''

Now, we could restrict ourselves entirely to internal PY_CFLAGS_NODIST,
but I think exposing the feature is really useful if users or
distributors want to specify optimizations, FPU behavior or other
things that should not generally show up in distutils.

[ned-deily]

I don't have a *really* strong opinion against it. It's just that I find the current plethora of configuration flags to be non-intuitive and confusing (and there are plenty of open issues agreeing with that sentiment) and adding another with the name CFLAGS_NODIST doesn't help. But, again, short of someone going in and doing a radical simplification of the whole configure.ac, Makefile.pre.in, and setup.py tangle, I guess exposing one more variable isn't going to make matters that much worse than they already are and it does solve a real problem. (Sorry to vent on your patch, Stefan.)

[ronaldoussoren]

I share Ned's sentiment on this. The amount of information exposed through sysconfig is too large as it is because a lot of that information is only useful during the built of python itself. I'm pretty sure that have been patches in the past where users tried to use some of those variables and were surprised they didn't work for an installed python.

Anyways, the patch looks good.

[pitrou]

This patch should definitely go in.

[python-dev]

New changeset 2a3538f14948 by Benjamin Peterson in branch '3.4':
add -Werror=declaration-after-statement only to stdlib extension modules (closes bpo-21121)
http://hg.python.org/cpython/rev/2a3538f14948

New changeset a5368cfbea0e by Benjamin Peterson in branch 'default':
merge 3.4 (bpo-21121)
http://hg.python.org/cpython/rev/a5368cfbea0e