[BUG] Closing and restarting a SparkSession can remove valid blocks in RapidsBufferStore

[abellina]

The issue reported here #4664 shows an underlying design issue with the singleton RapidsBufferStore.

In the test, the SparkSession was being stopped to prevent other problems (see #4669), but the underlying problem still remains: Is calling .close() on a session and creating a new session supported by Spark?

For shuffle specifically, when .close() is called on the session the SparkContext is also stopped, which resets the shuffle ids used and possibly ids for other blocks I haven't looked into (like broadcast blocks). The issue appears to be that stopping the context doesn't mean that unregisterShuffle calls are getting issued synchronously (hence the bug reported in #4664, because otherwise the store's block would have been removed). So there is some state that isn't clean when the context .close() returns, and there could be a late unregisterShuffle.

At this stage what we think should happen is that RapidsBufferStore should not be a singleton and somehow tied to the context. So that an unregisterShuffle (or other cleanup call) can be ignored for a store that was meant for a new context.