Work around out of memory issue with foreign allocator. #199
Conversation
app/src/handle-request.lisp
Outdated
| (yason:encode t s)))))))) | ||
|
|
||
| #+sbcl | ||
| (sb-ext:gc :full t)) |
There was a problem hiding this comment.
qvm-app already depends on trivial-garbage, so maybe trivial-garbage:gc instead?
app/src/handle-request.lisp
Outdated
| (yason:encode t s))))))))) | ||
| (yason:encode t s)))))))) | ||
|
|
||
| (tg:gc :full t)) |
There was a problem hiding this comment.
Maybe leave a comment indicating why this is required
|
Maybe we should explicitly deallocate instead of relying on the GC? It's scary that every request calls a GC which locks all threads. |
Right now this is only triggered when using foreign memory allocation and most users will be resorting to lisp allocation for the foreseeable future. I agree with you but I think the current patch is OK as a transient fix and I will let someone else (perhaps my future, not-on-vacation self?) produce a more definitive fix. |
|
I'll accept it if you write somewhere that it's a transient fix in the source. |
|
Let me give this another go later before merging. |
|
Happened upon this paper today which touches on some related GC themes: Dynamic Optimizations for SBCL Garbage Collection The paper is short and light on details, but all three of the optimizations mentioned seem like things qvm-app might potentially benefit from (with the huge caveat that the methods hinted at in the paper rely on running inside a container--though it isn't immediately clear that they wouldn't/couldn't work in a non-containerized environment). Relevant snippet from the paper:
In the paper, this is paired with a couple of other optimizations, one of which is to delay promoting objects out of the nursery if GC happens while an application request is in process. In the case of qvm-app, this would presumably mean that the QVM object remains in gen0 until after the request is finished, at which point it would no longer be live and get collected.
For the record, I'm not proposing we implement any of these for qvm-app. I just happened on the paper and the theme was topical/seemed worth sharing. Maybe stylewarning even remembers the talk and/or got a t-shirt 👕. |
Thanks for referencing that paper. I have read it and agree that it could be useful. The problem here is indeed that the GC is unaware of the true extent of a QVM object (in particular, its foreign-allocated vector of amplitudes) and that the QVM object survives for many generations. Triggering the GC based on the current process' resident segment size as suggested in the paper is a good solution. It is, however, unportable across Lisp implementations and operating systems. For instance, SBCL can use GENCGC on some architectures and CheneyGC on others whereas ECL uses the Boehm garbage collector. Also, the typical way to query available memory in Linux is via the proc filesystem whereas the mechanism differs in macOS (and Windows). One could create a thread that periodically monitors memory usage and manually triggers a full GC when necessary but I think a simpler solution to fix this properly is to manually free the amplitudes vector as soon as possible after finishing the request. |
Yep, I agree. |
I concur. |
🤝 |
The garbage collection added at the end of HANDLE-REQUEST in #199 prevents HANDLE-REQUEST from returning a value. Some of the handlers write to the response stream directly, but others just return a value, which hunchentoot handles sending back to the client.
The garbage collection added at the end of HANDLE-POST-REQUEST in #199 prevents HANDLE-POST-REQUEST from returning a value. Some of the handlers write to the response stream directly, but others just return a value and let HUNCHENTOOT handle sending the response back to the client. This commit (1) ensures that a response value is always returned from HANDLE-POST-REQUEST, and (2) ensures that cleanup / gc is always run, even if the request signals an error.
The garbage collection added at the end of HANDLE-POST-REQUEST in #199 prevents HANDLE-POST-REQUEST from returning a value. Some of the handlers write to the response stream directly, but others just return a value and let HUNCHENTOOT handle sending the response back to the client. This commit (1) ensures that a response value is always returned from HANDLE-POST-REQUEST, and (2) ensures that cleanup / gc is always run, even if the request signals an error.
Work around for #198.