Prototype Linearized Matrix

[clokep]

Prototype the Internet Draft draft-ralston-mimi-linearized-matrix as a proposal for the MIMI working group. see the list of differences from current Matrix.

Additionally we wish to demonstrate the interoperability of a DAG-capable server interconnecting with a LInearized Matrix network.

Done status: interoperable with the eigen-server from scratch implementation of Linearized Matrix:

1. Synapse should be able to act as a hub and pass events to eigen-server acting as a participant
   • Inviting a user on eigen-server.
   • Sending messages to a user on eigen-server.
   • Receiving messages from a user on eigen-server.
   • Sending state events to a user on eigen-server.
   • Receiving state events from a user on eigen-server.
2. Synapse should act as a link between a Linearized Matrix network and the wider Matrix network.
   • Events from a DAG should be linearized and sent to eigen-server
   • (TODO)

[clokep]

The initial v00 of the draft was partially implemented:

• Add a new room version
  • Update room version to fix oddities (see MSC2174, MSC2175, MSC2176, MSC3989)
• Add a new event format
• Update hashing algorithm to handle events sent from a hub
• Implement /_matrix/linearized endpoints

Work on this stopped once the I-D was updated to v02. Initial commits in the clokep/lm branch were for v00.

[clokep]

Current status is updating this for v02 of the I-D, this is happening on the same clokep/lm branch as before. (Note that I've been rebasing this branch so it likely isn't the best to track.)

My overall goal is to get things working with it being OK if it is a bit hacky. (I.e. I'm not planning for this branch to be a pull request.) Once we're confident in the design and the proposed I-D then we'll want to break this branch up into a bunch of PRs with associated MSCs.

---

• Update the room version to be based on room v11 (MSC3820)
  • Disable restricted and knock-restricted rooms
  • Disable third-party invites (see Validate third_party_invite on membership invites according to v10 auth rules eigen-server#4)
  • Stabilize room v11: Support room version 11 #15912
  • Double check that size limits are strictly enforced
  • Improve validations of e.g. Matrix IDs, room IDs, and state keys to enforce the I-D
  • Remove notifications.* checks from power levels in auth rules
  • RFC8785 for canonical JSON
  • Update content hash
  • Do not allow IPv4/IPv6 servers
• Track if other servers are Linearized-only
• Update the event format
  • Remove depth and unsigned on events for LM
  • Add an optional hub_server field
• Handle content hashes for LPDUs
• Remove references to m.room.hub events
• Implement new endpoints:
  • /invite
  • /send
  • /event
  • /backfill
  • /send_join
  • /send_leave
  • /send_knock
  • /device
  • Properly handle transaction IDs
  • Update 404 semantics for unknown vs unviewable events
• Call new endpoints
  • /invite
  • /send
  • /event
  • /backfill
  • /send_join
  • /send_leave
  • /send_knock
  • /device
• Handle new EDU format
  • m.device_list_update
  • m.direct_to_device
• Accept LPDUs
  • /send
  • /send_join
  • /send_leave
  • /send_knock
• Return LPDUs
  • /make_join
  • /make_leave
  • /make_knock
• Fan-out PDUs to any participant servers (i.e. act like a hub)
• Forward invites between participants (see New invite flow (blocking send/receive) eigen-server#33)
• Fix-up device handling logic
• Deal with MLS logic

[clokep]

The testing steps I've been using for this are:

1. Checkout clokep/lm
2. Run a local Synapse (mine is at 8480 using the demo scripts): python -m synapse.app.homeserver --config-path demo/8080/8080.config
3. Checkout eigen-server and get that setup.
4. Run eigen-server ES_PORT=3000 ES_SIGNING_KEY_PATH=./p3000.signing.key yarn dev:server
5. Run eigen-server's client: ES_PORT=3000 yarn dev:client:nobuild (must wait for step 4 first)
6. Open Element Web
7. Register/login (@alice:localhost:8480 for me).
8. Create an (unencrypted) room
9. Upgrade the room to get the proper version: /upgraderoom org.matrix.i-d.ralston-mimi-linearized-matrix.02 in Element Web
10. Join the new room
11. Invite the remote user (eigen-server prints out your username when you start the client): /invite @<uuid>:localhost:3000
12. In the eigen-server client accept the invite with /join <room ID>
13. Start chatting

[turt2live]

backref for #15962 and #15963 as "discovered during LM interop testing". They are both fixed on the clokep/lm branch, without tests.

[clokep]

I think I'm going to call this "done" for now until a new draft of MIMI comes out.

You can see results in an This Week in Matrix from August 4, 2023.