APIs, semantics and strategies for ergonomically reducing allocations

[AriMKatz]

From Slack:

Naive question, how far can we get by just going all out on immutability with things like https://github.com/JuliaObjects/Accessors.jl, the new immutable array PR? and JuliaLang/julia#11902 (comment) , https://github.com/JuliaFolds/BangBang.jl and TKF's https://github.com/tkf/Mutabilities.jl ?

https://github.com/aviatesk/EscapeAnalysis.jl Should provide analysis to help reason about allocations. That could be fed to custom compiler plugins with the above APIs to elide allocations. Then JET as in #54 could help users squeeze out the last bits left

Edit:

Also https://github.com/RelationalAI-oss/Blobs.jl and https://github.com/JuliaSIMD/ManualMemory.jl

[brenhinkeller]

It's a good question -- possibly quite a bit? If everything is on the stack (alloca'd), which immutable things usually are in Julia, then there should be no problem other than having to GC.@preserve before calling out to any c/llvm function with a pointer to that memory.

Using the ManualMemory.j trick, you can even have these stack-allocated objects be mutable (e.g.: https://julialang.zulipchat.com/#narrow/stream/137791-general/topic/StaticCompiler.2Ejl/near/270324640). The only limitation is that they have to be statically-sized (i.e., size, or at least an upper bound on size, known at compile time). In principle, you might be able to get this via other paths as well.

The other handy thing, if you really want dynamically-sized arrays, is that it's totally fine to directly Libc.malloc yourself some space on the heap manually. There are a few examples of both of these tricks, among other handy things (like not having to redefine IO primitives like puts and printf in order to get non-allocating IO) in https://github.com/brenhinkeller/StaticTools.jl

[brenhinkeller]

Oh, looks like this may be slightly moot now! https://julialang.zulipchat.com/#narrow/stream/256674-compiler-plugins/topic/StaticCompiler.2Ejl/near/271385050 (edit: PR #58 )

Eliminating GC allocations is still my passion project though 😆

[AriMKatz]

Only slightly moot, because there are still use cases for totally eliminating the runtime, so I morally support your passion project!

Also I was misunderstanding the purpose of immutable arrays. Immutability in Julia is conflated with stack allocations, and I assumed the same with arrays. It's more the static sizing that's helpful apparently

[brenhinkeller]

Yeah, the way Julia does things currently definitely leads to the conflation of immutability and stack allocation. In principle you don't really even need things to be statically-sized to be stack-allocated (LLVM itself supports dynamically sized allocas), there's just currently no way to get dynamically-sized allocas in Julia, even using fancy tricks. Some more discussion here: https://julialang.zulipchat.com/#narrow/stream/256674-compiler-plugins/topic/StaticCompiler.2Ejl'd.20standalone.20Hello.20World/near/270904534

[chriselrod]

there's just currently no way to get dynamically-sized allocas in Julia, even using fancy tricks.

@tkf gave an example of a trick to get one, but it involved passing a ccallable Julia function pointer (that takes the alloca as an argument) into an llvmcall, which has some overhead.