leak in xsub0_test during socket close

[gdamore]

In CI tests...

==7809==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 112 byte(s) in 1 object(s) allocated from:
    #0 0x5614ca087898 in __interceptor_calloc (/home/runner/work/nng/nng/build/src/sp/protocol/pubsub0/xsub_test+0xc9898) (BuildId: 2381fbf51f90e39c574aad78e6c7e32f9a1947ba)
    #1 0x5614ca10d7f4 in nni_zalloc /home/runner/work/nng/nng/src/platform/posix/posix_alloc.c:26:19
    #2 0x5614ca0f3459 in nni_msg_alloc /home/runner/work/nng/nng/src/core/message.c:378:11
    #3 0x5614ca0d036c in nng_msg_alloc /home/runner/work/nng/nng/src/nng.c:1412:10
    #4 0x5614ca0d0172 in nng_send /home/runner/work/nng/nng/src/nng.c:160:12
    #5 0x5614ca0ccbd6 in test_xsub_close_during_recv /home/runner/work/nng/nng/src/sp/protocol/pubsub0/xsub_test.c:308:3
    #6 0x5614ca0ce956 in test_do_run_ /home/runner/work/nng/nng/src/testing/acutest.h:1026:9
    #7 0x5614ca0c6629 in test_run_ /home/runner/work/nng/nng/src/testing/acutest.h:1122:23
    #8 0x5614ca0c3cb9 in main /home/runner/work/nng/nng/src/testing/acutest.h:1724:13
    #9 0x7f5e0f029d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: a43bfc8428df6623cd498c9c0caeb91aec9be4f9)

Indirect leak of 67 byte(s) in 1 object(s) allocated from:
    #0 0x5614ca087898 in __interceptor_calloc (/home/runner/work/nng/nng/build/src/sp/protocol/pubsub0/xsub_test+0xc9898) (BuildId: 2381fbf51f90e39c574aad78e6c7e32f9a1947ba)
    #1 0x5614ca10d7f4 in nni_zalloc /home/runner/work/nng/nng/src/platform/posix/posix_alloc.c:26:19
    #2 0x5614ca0f3ca9 in nni_chunk_grow /home/runner/work/nng/nng/src/core/message.c:149:17
    #3 0x5614ca0f34bb in nni_msg_alloc /home/runner/work/nng/nng/src/core/message.c:388:8
    #4 0x5614ca0d036c in nng_msg_alloc /home/runner/work/nng/nng/src/nng.c:1412:10
    #5 0x5614ca0d0172 in nng_send /home/runner/work/nng/nng/src/nng.c:160:12
    #6 0x5614ca0ccbd6 in test_xsub_close_during_recv /home/runner/work/nng/nng/src/sp/protocol/pubsub0/xsub_test.c:308:3
    #7 0x5614ca0ce956 in test_do_run_ /home/runner/work/nng/nng/src/testing/acutest.h:1026:9
    #8 0x5614ca0c6629 in test_run_ /home/runner/work/nng/nng/src/testing/acutest.h:1122:23
    #9 0x5614ca0c3cb9 in main /home/runner/work/nng/nng/src/testing/acutest.h:1724:13
    #10 0x7f5e0f029d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: a43bfc8428df6623cd498c9c0caeb91aec9be4f9)

SUMMARY: AddressSanitizer: 179 byte(s) leaked in 2 allocation(s).
Test xsub close during pipe recv:
  xsub_test.c:320: Check nng_sub0_open_raw(&sub) succeeds... ok
  xsub_test.c:321: Check nng_pub0_open(&pub) succeeds... ok
  xsub_test.c:322: Check nng_socket_set_ms(sub, NNG_OPT_RECVTIMEO, 1000) succeeds... ok
  xsub_test.c:323: Check nng_socket_set_ms(pub, NNG_OPT_SENDTIMEO, 100) succeeds... ok
  xsub_test.c:324: Check nng_socket_set_int(sub, NNG_OPT_RECVBUF, 5) succeeds... ok
  xsub_test.c:325: Check nng_socket_set_int(pub, NNG_OPT_SENDBUF, 20) succeeds... ok
  xsub_test.c:327: Check nuts_marry(pub, sub) succeeds... ok
  xsub_test.c:337: Check nng_close(sub) succeeds... ok
  SUCCESS: All conditions have passed.
  Duration: 0.112590 secs

I believe that either when we are closing the xpub0 socket, or the xsub0 socket, we are not cleaning things properly. This is probably in the upper queues. We need to be sure that if we discard the queues, that we discard the messages in them.

[gdamore]

I believe that the problem here is that the protocols that use transport pipe_send can wind up losing the message if the pipe has been closed.

This is because the transport uses nng_aio_begin (checking for success), but doesn't complete the aio (we need to not call the back for SP transports in this case, or we would be an infinite loop).

This means that there is no feedback mechanism for the protocol to discard the message. But this doesn't matter really. We can at this point just safely discard the message -- the protocol doesn't need to do anything with it (and indeed will not).

[gdamore]

This affects all protocols, and is a bug in the transports.