Question: Pub/Sub wrt Memory Mapped Buffers

[goglusid]

I'm using the C++ API.

If an application has either an IPC Publication and an IPC Subscription or a Network Publication and a Spy Subscription, then the Publisher's term buffers get mapped twice.

Given that an IPC Subscription and a Spy Subscription directly reads the Publication's Term buffers, why are they mapped twice?

Also would it be feasible to only map the Publisher's term buffers once or this would "completely break" the design?

[mjpt777]

What do you mean by mapped twice? Is that the once on the client and once in the driver or do you mean something else?

[mjpt777]

If it is to do with them being mapped once for the publication and then once for the subscription in the client then this is because they can have difference lifetimes and even be in different processes.

[goglusid]

Sorry for the misunderstanding and thanks for the fast reply.

Ex:
TermBufferSize: 8 MB
Process#1: Creates an 2 IPC Publications on stream 10
Process#2: Creates 2 IPC Subscriptions on stream 10.
Process#3: Creates 4 IPC Subscriptions on stream 10.

In this example:

Process#1: Has a working set memory size: ~29 MB
Process#2: Has a working set memory size: ~54 MB
Process#3: Has a working set memory size: ~103 MB

Given that an IPC Subscription is, from what I understand, a sequence id on top of a publisher's term buffers, then why are we mapping these term buffers for each IPC subscriptions?

Moreover, we already do this optimisation for the 2 IPC Publications in Process#1.

[goglusid]

In this example, the MediaDriver as a working set memory size: ~109 MB. Meaning it also maps the same term buffers multiple times.

[mjpt777]

The publications are concurrent so are treated differently. We could save a mapping on the subscriber side but is it really worth it? Consider that the memory is mapped, it is the same memory just under a different mapping. It is not like more physical memory is required.

[goglusid]

I'm running in a process compiled in 32 Bits. On Windows, when using the large address space process flag, I can have a max of 4 GB of application space. My current understanding is that each mapping subtracts from this 4 GB limit. Also since Subscription aren't thread safe, I need one for each thread within that process. Add to this the fact that I'm planning to have a lot of remote hosts meaning a term buffer per host. I would say it is worth it but maybe only for my case.

My takeaway is that: "We could save a mapping on the subscriber side " So if I need to go there to save virtual address space then I will do it.

Regards and Thanks for taking the time to respond. ;)

[mjpt777]

The mappings in the driver cannot be reduced as the absolutely essential. The only potential saving is the buffers in the client for subscriptions. I can take a look at how much effort that would be consolidate and ref count. It would save on TLB entries and maybe help the Archiver.

I feel sorry for you on 32-bit Windows :-) You will also pay a significant performance penalty for that.