Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: recursive mutex implementation #5731

Merged
merged 3 commits into from
Feb 21, 2017
Merged

Conversation

melshuber
Copy link

This recursive Mute implementation is derived from the non-recursive variant.
The implementation is basically a port of
https://github.com/RIOT-OS/RIOT/pull/4529/files#diff-8f48e1b9ed7a0a48d0c686a87cc5084eR35.

@miri64
Copy link
Member

miri64 commented Aug 5, 2016

Why not just have a function mutex_rec_lock() or something like that (as discussed in 5656)? This would take a lot of code duplication out of #5656.

@miri64
Copy link
Member

miri64 commented Aug 5, 2016

Alternatively make it an optional module in sys/. It's better to keep core small.

@miri64 miri64 added Area: core Area: RIOT kernel. Handle PRs marked with this with care! Type: new feature The issue requests / The PR implemements a new feature for RIOT labels Aug 5, 2016
@melshuber
Copy link
Author

I added it to core because of the discussion in #5656, but if sys is preferable, i can move it as well.

@melshuber
Copy link
Author

mutex_rec_lock is not possible because a mutex_t consists solely of an integer. The rmutex_t implementation needs additional memory (refcount and owner). code duplication is avoided be reusing the existing mutex implementation.

@miri64
Copy link
Member

miri64 commented Aug 5, 2016

Arghs, sorry >.<! Should have examined the code more carefully. Might be okay to have this in core after all.

@miri64
Copy link
Member

miri64 commented Aug 5, 2016

(I leave this decision to the assigned reviewers though).

@miri64 miri64 added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Aug 5, 2016
core/rmutex.c Outdated
kernel_pid_t owner;
/* try to lock the mutex */
DEBUG("rmutex %" PRIi16" : trylock\n", thread_getpid());
switch (mutex_trylock(&rmutex->mutex)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use if (… == 0) { instead of switch with a fall-through.

@Kijewski Kijewski self-assigned this Aug 5, 2016
@Kijewski
Copy link
Contributor

Kijewski commented Aug 5, 2016

IMO the implementation is small and useful enough to go into mutex.c directly. Most is not all platforms link with --gc-sections by now, so we don't need to fear bloated binaries.

@Kijewski Kijewski removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Aug 5, 2016
@miri64
Copy link
Member

miri64 commented Aug 8, 2016

IMO the implementation is small and useful enough to go into mutex.c directly. Most is not all platforms link with --gc-sections by now, so we don't need to fear bloated binaries.

Well it still uses another type.

@jnohlgard
Copy link
Member

It seems like some comments by @miri64 on this PR that I received via email disappeared?

Anyway, regarding atomic accesses, please use atomic_int, alternatively, with #5688 applied, use atomic_load/atomic_store (http://en.cppreference.com/w/c/atomic/atomic_load).
With #5688, the compiler will ensure that optimized atomic operations are used when possible, with a fall back to library implementations if the hardware does not support it.

@miri64
Copy link
Member

miri64 commented Aug 8, 2016

@gebart you mean the one above: #5731 (comment)

@jnohlgard
Copy link
Member

@miri64 yes, I guess it was just temporary blindness then

@melshuber
Copy link
Author

I posted an update using atomic_int_least16_t. least16 because kernel_pid_t is an int16_t.
Is there a better way to infer the atomic type based on kernel_pid_t?
I think using atomic_int unnecessarily wastes resources.

I think memory_order_relaxed should be enough, but it would be great if anyone else can confirm this.

@jnohlgard
Copy link
Member

Did you try _Atomic(kernel_pid_t)?

@melshuber
Copy link
Author

_Atomic(kernel_pid_t) works on some boards

> $ BOARD=stm32f3discovery make -C tests/rmutex/
> make: Entering directory '/home/melshuber/Work/riot.bak/tests/rmutex'
> Building application "rmutex" for "stm32f3discovery" with MCU "stm32f3".
> 
> "make" -C /home/melshuber/Work/riot.bak/boards/stm32f3discovery
> "make" -C /home/melshuber/Work/riot.bak/core
> "make" -C /home/melshuber/Work/riot.bak/cpu/stm32f3
> "make" -C /home/melshuber/Work/riot.bak/cpu/cortexm_common
> "make" -C /home/melshuber/Work/riot.bak/cpu/stm32_common
> "make" -C /home/melshuber/Work/riot.bak/cpu/stm32_common/periph
> "make" -C /home/melshuber/Work/riot.bak/cpu/stm32f3/periph
> "make" -C /home/melshuber/Work/riot.bak/drivers
> "make" -C /home/melshuber/Work/riot.bak/drivers/periph_common
> "make" -C /home/melshuber/Work/riot.bak/sys
> "make" -C /home/melshuber/Work/riot.bak/sys/newlib
> "make" -C /home/melshuber/Work/riot.bak/sys/tsrb
> "make" -C /home/melshuber/Work/riot.bak/sys/uart_stdio
> "make" -C /home/melshuber/Work/riot.bak/sys/xtimer
>    text      data     bss     dec     hex filename
>    9012       128    2788   11928    2e98 /home/melshuber/Work/riot.bak/tests/rmutex/bin/stm32f3discovery/rmutex.elf
> make: Leaving directory '/home/melshuber/Work/riot.bak/tests/rmutex'
> $ BOARD=native make -C tests/rmutex/
> make: Entering directory '/home/melshuber/Work/riot.bak/tests/rmutex'
> Building application "rmutex" for "native" with MCU "native".
> 
> In file included from /home/melshuber/Work/riot.bak/tests/rmutex/main.c:25:0:
> /home/melshuber/Work/riot.bak/core/include/rmutex.h:58:5: error: ISO C99 does not support the ‘_Atomic’ qualifier [-Werror=pedantic]
>      _Atomic(kernel_pid_t) owner;
>      ^
> cc1: all warnings being treated as errors
> /home/melshuber/Work/riot.bak/Makefile.base:60: recipe for target '/home/melshuber/Work/riot.bak/tests/rmutex/bin/native/rmutex/main.o' failed
> make[1]: *** [/home/melshuber/Work/riot.bak/tests/rmutex/bin/native/rmutex/main.o] Error 1
> /home/melshuber/Work/riot.bak/tests/rmutex/../../Makefile.include:260: recipe for target 'all' failed
> make: *** [all] Error 2
> make: Leaving directory '/home/melshuber/Work/riot.bak/tests/rmutex'
> 
> $ CFLAGS=-std=c11 BOARD=native make -C tests/rmutex/
> make: Entering directory '/home/melshuber/Work/riot.bak/tests/rmutex'
> Building application "rmutex" for "native" with MCU "native".
> 
> "make" -C /home/melshuber/Work/riot.bak/boards/native
> In file included from /usr/include/ucontext.h:26:0,
>                  from /home/melshuber/Work/riot.bak/cpu/native/include/native_internal.h:39,
>                  from /home/melshuber/Work/riot.bak/boards/native/board_config.c:25:
> /usr/include/sys/ucontext.h:238:5: error: unknown type name ‘stack_t’
>      stack_t uc_stack;
>      ^
> In file included from /home/melshuber/Work/riot.bak/boards/native/board_config.c:25:0:
> /home/melshuber/Work/riot.bak/cpu/native/include/native_internal.h:88:41: error: ‘struct addrinfo’ declared inside parameter list [-Werror]
>  extern void (*real_freeaddrinfo)(struct addrinfo *res);
>                                          ^
> /home/melshuber/Work/riot.bak/cpu/native/include/native_internal.h:88:41: error: its scope is only this definition or declaration, which is probably not what you want [-Werror]
> /home/melshuber/Work/riot.bak/cpu/native/include/native_internal.h:143:25: error: ‘SIGSTKSZ’ undeclared here (not in a function)
>  extern char __isr_stack[SIGSTKSZ];
>                          ^
> cc1: all warnings being treated as errors
> /home/melshuber/Work/riot.bak/Makefile.base:60: recipe for target '/home/melshuber/Work/riot.bak/tests/rmutex/bin/native/board/board_config.o' failed
> make[2]: *** [/home/melshuber/Work/riot.bak/tests/rmutex/bin/native/board/board_config.o] Error 1
> /home/melshuber/Work/riot.bak/Makefile.base:20: recipe for target 'ALL--/home/melshuber/Work/riot.bak/boards/native' failed
> make[1]: *** [ALL--/home/melshuber/Work/riot.bak/boards/native] Error 2
> /home/melshuber/Work/riot.bak/tests/rmutex/../../Makefile.include:260: recipe for target 'all' failed
> make: *** [all] Error 2
> make: Leaving directory '/home/melshuber/Work/riot.bak/tests/rmutex'
> $ gcc -v
> Using built-in specs.
> COLLECT_GCC=gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Debian 4.9.2-10' --with-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --with-arch-32=i586 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 4.9.2 (Debian 4.9.2-10)
> $ arm-none-eabi-gcc -v
> Using built-in specs.
> COLLECT_GCC=arm-none-eabi-gcc
> COLLECT_LTO_WRAPPER=/opt/toolroot/arm/gcc-arm-none-eabi-4_9-2015q1/bin/../lib/gcc/arm-none-eabi/4.9.3/lto-wrapper
> Target: arm-none-eabi
> Configured with: /home/build/work/GCC-4-9-build/src/gcc/configure --target=arm-none-eabi --prefix=/home/build/work/GCC-4-9-build/install-native --libexecdir=/home/build/work/GCC-4-9-build/install-native/lib --infodir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/home/build/work/GCC-4-9-build/install-native/arm-none-eabi --build=i686-linux-gnu --host=i686-linux-gnu --with-gmp=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-mpfr=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-mpc=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-isl=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-cloog=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-libelf=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib-list=armv6-m,armv7-m,armv7e-m,cortex-m7,armv7-r
> Thread model: single
> gcc version 4.9.3 20150303 (release) [ARM/embedded-4_9-branch revision 221220] (GNU Tools for ARM Embedded Processors)
> 

core/rmutex.c Outdated
rmutex->refcount--;

/* check if we still hold the mutex */
if (rmutex->refcount==0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spaces before and after ==

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@jnohlgard
Copy link
Member

I guess -pedantic gets in the way here. We also can't switch to C11 until the MSP430's are updated with to less ancient toolchain. (yes, 2012 is ancient when you look at the history of RIOT)

To fix your error when adding CFLAGS you could try -std=gnu11 instead, I think that will fix the missing typedefs.

@melshuber
Copy link
Author

melshuber commented Aug 9, 2016

Ok with gnu11 it works on the native board but not at stmf3discovery board.
But if we want to merge this, i think it is a bad idea to let the user fiddle around -std=XXXX flags

@melshuber
Copy link
Author

I think we agreed to use an atomic_XXX_t for owner. But I am still not sure which one is best in this case. I tend to atomic_least16_t as it should be compatible with all boards.
Once MSP430 is updated it can be changed later to _Atomic(kernel_pid_t) later on.

core/rmutex.c Outdated
/* mutex is already held
*
* Case 1: the current thread holds the mutex
* Invariant 1: holds
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing something, but what invariant are you referring to here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invariant is an incorrect word i think.

the point is that the outcome of the condition
if ( owner != thread_getpid() )
well never change even if rmutex->owner is changed by an other thread (holder of the mutex)
i will update the comment occrdingly

@OlegHahm
Copy link
Member

I think ifdef CPU_MSP430 and ifdef __WITH_AVRLIBC__ would work,

@OlegHahm
Copy link
Member

(Not very elegant, agreed...)

@melshuber
Copy link
Author

insn't there something like
ifdef SUPPORTS_STDATOMC ir similar

@OlegHahm
Copy link
Member

Have you tried HAVE_BUILTIN_ATOMIC?

@OlegHahm
Copy link
Member

You could also check for the compiler version, I guess.

@melshuber
Copy link
Author

OFF Topic:

I think i just found an issue in ATOMIC_VALUE on avr8

#include <atomic.h>

atomic_int_t atomic = ATOMIC_INIT(0);

int atomic_val(void) {
    return ATOMIC_VALUE(atomic);
}

int main(void)
{
    while (1);
}

copiles to

$ BOARD=arduino-mega2560 make
$ avr-objdump -S bin/arduino-mega2560/atomic/main.o

Disassembly of section .text.atomic_val:

00000000 <atomic_val>:
#include <atomic.h>

atomic_int_t atomic = ATOMIC_INIT(0);

int atomic_val(void) {
    return ATOMIC_VALUE(atomic);
   0:	80 91 00 00 	lds	r24, 0x0000
   4:	90 91 00 00 	lds	r25, 0x0000
}
   8:	08 95       	ret

Disassembly of section .text.startup.main:

00000000 <main>:

int main(void)
{
   0:	00 c0       	rjmp	.+0      	; 0x2 <__zero_reg__+0x1>

you see two lds instructions in atomic_val.c

Problem case: Suppose a higher priority thread interrupts just after the first lds and changes the value itself to 0xFFFF. In this case an inconsistent value would be returned (endianness depended either 0xFF or 0xFF00)

@melshuber
Copy link
Author

to avoid doing work twice i tell you what i will do:

  • i change the implementation for msp430 and avr8 to use atomic_int_t
  • however i cannot do any functional tests because a do not have any targets for this cpus, however i can run the tests on an cortex-m3 mcu, using atomic_int_t for the tests

@OlegHahm
Copy link
Member

OlegHahm commented Jan 26, 2017

to avoid doing work twice i tell you what i will do:

i change the implementation for msp430 and avr8 to use atomic_int_t
however i cannot do any functional tests because a do not have any targets for this cpus,
however i can run the tests on an cortex-m3 mcu, using atomic_int_t for the tests

Sounds like a good plan to me. I can test on MSP430.

@OlegHahm
Copy link
Member

Regarding the AVR problem: could you open an issue, please?

@PeterKietzmann
Copy link
Member

@melshuber I will move the the milestone label now as we will 'feature freeze' in about the next minutes. That must not mean that your PR can't be merged to master once approved and built correctly.

@PeterKietzmann PeterKietzmann modified the milestones: Release 2017.04, Release 2017.01 Jan 26, 2017
@melshuber
Copy link
Author

@OlegHahm: shall we wait for #5688 because it might fix the build issues at least fro msp430

@OlegHahm
Copy link
Member

Seems reasonable.

@melshuber
Copy link
Author

Now that #5688 has been merged i think most of the problems
from Jenkins and Murdok are gone.

However before i rebase i'd like to address the other two issues left in #5731 (comment).

Bullet 1) static-tests - I have still no idea how to fix it?
Bullet 2) cortex_m0_1: is my solution acceptable?

@kaspar030
Copy link
Contributor

Bullet 1) static-tests - I have still no idea how to fix it?

The error pops up when at the start of the build, the "needs squashing" label is set. It isn't anymore.

Bullet 2) cortex_m0_1: is my solution acceptable?

You mean removing the board with little memory? that's perfect!

In contrast to normal mutexes, reeentrant mutexes allow to be relocked
multiple times from the same thread.
* fixed BOARD_INSUFFICIENT_MEMORY for rmutex
@melshuber melshuber force-pushed the master branch 2 times, most recently from e083cf6 to b9bb22b Compare February 9, 2017 14:34
@melshuber
Copy link
Author

Sorry, i accidently pushed some garbage, it should be already fixed - hower you might want to retrigger jenkins

core/rmutex.c Outdated
* Condition 2: holds
* rmutex->owner == thread_getpid()
*
* Note for Case 1:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the note is for Case 2?

core/rmutex.c Outdated
kernel_pid_t owner;

/* try to lock the mutex */
if (mutex_trylock(&rmutex->mutex) == 0) {
Copy link
Member

@smlng smlng Feb 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some code deduplication possible here, you could make an internal function like

int _rmutex_lock(rmutex_t *rmutex, bool try)
{
    kernel_pid_t owner;
    if (mutex_trylock(&rmutex->mutex) == 0) {
        owner = atomic_load_explicit( &rmutex->owner, memory_order_relaxed);
        if ( owner != thread_getpid() ) {
            if (try) {
                return 0;
            }
            mutex_lock(&rmutex->mutex);
        }
/* and so on */
}

void rmutex_lock(rmutex_t *rmutex)
{
    _rmutex_lock(rmutex, false);
}

int rmutex_trylock(rmutex_t *rmutex)
{
    return _rmutex_lock(rmutex, true);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see also here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points, I aggree.
Do you prefer a the update squashed or as separate commit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please make the update into a separate commit for reviewing first, but it should be squashed before merge.

Copy link
Member

@smlng smlng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my earlier comments, please deduplicate code as done in the existing mutex implementation for functions rmutex_lock and rmutex_trylock

@smlng
Copy link
Member

smlng commented Feb 21, 2017

looks good now, tests works: ACK. Still, I'd rather leave the final merge to someone more familiar with RIOT core ...

void rmutex_unlock(rmutex_t *rmutex)
{
assert(atomic_load_explicit(&rmutex->owner,memory_order_relaxed) == thread_getpid());
assert(rmutex->refcount > 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late commenting, but I just stumbled over this:

This means calling rmutex_unlock() from as non-owner or with refcount==0 is undefined? Wouldn't it be safer to handle those cases somehow sane?

E.g., after calling rmutex_unlock(), the mutex is guarantueed to be free. That would mean, if non-owner calls, it needs to lock first. If the owner calls but refcount==0, the function just returns.

What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked, C11 rmutexes have "The behavior is undefined if the mutex is not locked by the calling thread.", so I guess we can go with that. So let's postpone this discussion.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think calling unlock on without owning the lock is an error in the application, iff doing so the behaviour can be left undefined. We could argue to add a comment in the api description

@kaspar030
Copy link
Contributor

&go.

@kaspar030 kaspar030 merged commit 8d207ca into RIOT-OS:master Feb 21, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: core Area: RIOT kernel. Handle PRs marked with this with care! CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: new feature The issue requests / The PR implemements a new feature for RIOT
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants