Defensive code for alloc/dealloc during TLS teardown

[mjp41]

If an allocation or deallocation occurs during TLS teardown, then it is
possible for a new allocator to be created and then this is leaked. On
the mimalloc-bench mstressN benchmark this was observed leading to a
large memory leak.

This fix, detects if we are in the TLS teardown phase, and if so,
the calls to alloc or dealloc must return the allocator once they have
perform the specific operation.

[Add:] Also fixes unimplemented ABA primitive for ARM. Currently uses locking.

[Add:] Some debug infrastructure to deal with Posix platforms.

[mjp41]

Moved to draft due to failures on Arm Debug.

[mjp41]

I have addressed #50 with a locking based implementation. This should fix the ARM issues we were seeing.

[plietar]

LGTM, although I'm slightly concerned about the use of std::function. Creating them can result in an allocation, which could be bad if snmalloc is used as the default allocator.

We currently have few enough captures that it probably doesn't actually allocate (from what I can tell, GCC and clang can fit two words without allocation), but it feels pretty fragile.

Adding a template to InitThreadAllocator would help, but you can't have templated non type template arguments (ie. template<template<typename Fn> void* (*InitThreadAllocator)(const Fn&)>).

Instead we can wrap the two functions as static methods:

struct AllocInitializer {
  static bool needs_initialization() { [...] }
  template<typename Fn>
  static auto with_initialized_allocator(Fn& fn) {
    [...]
    auto value = fn(allocator);
    [...]
    return value;
  }
};

Then have Alloc parametrized over the concrete initializer instead and use it like:

template<typename Initializer, [...]>
struct Allocator {
  void* small_alloc_first_alloc(size_t size) {
    return Initializer::with_initialized_allocator([size](Allocator* allocator) {
      return allocator->alloc(size);
    });
  }
}

I think this is guaranteed to not have any allocation.

[mjp41]

LGTM, although I'm slightly concerned about the use of std::function. Creating them can result in an allocation, which could be bad if snmalloc is used as the default allocator.

We currently have few enough captures that it probably doesn't actually allocate (from what I can tell, GCC and clang can fit two words without allocation), but it feels pretty fragile.

Adding a template to InitThreadAllocator would help, but you can't have templated non type template arguments (ie. template<template<typename Fn> void* (*InitThreadAllocator)(const Fn&)>).

Instead we can wrap the two functions as static methods:

struct AllocInitializer {
  static bool needs_initialization() { [...] }
  template<typename Fn>
  static auto with_initialized_allocator(Fn& fn) {
    [...]
    auto value = fn(allocator);
    [...]
    return value;
  }
};

Then have Alloc parametrized over the concrete initializer instead and use it like:

template<typename Initializer, [...]>
struct Allocator {
  void* small_alloc_first_alloc(size_t size) {
    return Initializer::with_initialized_allocator([size](Allocator* allocator) {
      return allocator->alloc(size);
    });
  }
}

I think this is guaranteed to not have any allocation.

I really like this idea, I have not managed to get it to work with the current layering and header only nature AllocInitialiser needs to know about Alloc, and Alloc needs to know about AllocInitialiser. So that is a little problem.

I would like to not use std::function, and I think I can make this work. I propose I complete this PR, as it is already getting quite big, and submit this idea as a separate PR later.

[plietar]

I would like to not use std::function, and I think I can make this work. I propose I complete this PR, as it is already getting quite big, and submit this idea as a separate PR later.

Sure, sounds good.