This is a (WIP) toolkit for running multiplayer game servers on Edgegap, where the games are built with Bevy using the Lightyear networking library.
The goal is to have an easy-to-deploy system either yourself with docker-compose, or in the cloud, to use Edgegap to spin up gameservers on demand.
My testbed for this is bevygap-spaceships – a modified
stand-alone version of the spaceships
example from the Lightyear repo.
Is the live demo of bevygap-spaceships working? Hopefully!
Documentation by book: The Bevygap Book
mdbook serve -o book/
if you want to view it locally.
- Game client talks to
bevygap_matchmaker_httpd
via websocket, asking to play, and ultimately expecting a response including aConnectToken
and gameserverip:port
. - This makes a nats request to the matchmaker service
- An edgegap session is created, which corresponds to a gameserver instance (which may be autodeployed in response to this session creation)
- Matchmaker generates a Lightyear
ConnectToken
, and associates the token'sClientId
with the edgegap session ID in NATS KV. - Client uses the
ConnectToken
to connect to the gameserver - Gameserver maintains a list of active connections in NATS KV, which the matchmaker watches.
- When a client disconnects, the gameserver removes their entry from
active_connection
, causing the matchmaker to delete the Edgegap session.
Clients wishing to connect to the game make a request to our matchmaker service, which then:
- Creates an Edgegap session (which may trigger a new server deployment)
- Creates a new Lightyear client id and connect token, associated with the session
- Stores the token/session mapping in NATS KV
- Returns the connect token, and gameserver IP and port to the client. (the gameserver ip+port will be a machine controlled by Edgegap, running your game server's docker image)
- Deletes edgegap sessions when clients disconnect, by watching
active_connections
in NATS KV
An http endpoint to make "i want to play" requests to the matchmaker. The matchmaker itself only exposes a service to NATS, not http.
A bevy plugin for the game client, to replace the normal lightyear commands.connect_client()
call.
The new commands.bevygap_connect_client()
function will make a request to the matchmaker, then modify lightyear's config to set the supplied
game server socket address and connect token, then call commands.connect_client()
for you.
A bevy plugin for the gameserver, which loads its deployment context from the edgegap API on boot, and connects to our NATS instance in order to lookup session information.
Shared code for some protocol and NATS stuff, used between the matchmaker and gameserver.
NATS is the shared state and messaging backend between our various components.
Autogenerated client for the edgegap API using openapi-generator
. See gen-edgegap-client.sh
.
a basic bevy websocket client, used to talk to the matchmaker.
probably want to prefix all NATS subjects/buckets with the edgegap app name or something, so that multiple apps on edgegap can share the nats instance without conflicts.
should probably be restricted by the nats creds
https://w3c.github.io/webtransport/#dom-webtransportoptions-servercertificatehashes
https://w3c.github.io/webtransport/#verify-a-certificate-hash
Strict cert requirements, we autogenerate one on boot and report the digest to the client, so as long as the server doesn't stay up for more than 14 days, we're fine.
Chrome can't do CORS to http://localhost from http://lan-machine it seems, lan-machine would need to serve https. https://stackoverflow.com/questions/66534759/cors-error-on-request-to-localhost-dev-server-from-remote-site
Access to fetch at 'http://127.0.0.1:3000/wannaplay' from origin 'http://lan-machine:8008' has been blocked by CORS policy: The request client is not a secure context and the resource is in more-private address space local
.
add httpd to ly servers that respond like the MM, generating a cert etc? httpd example from bevy_remote: https://github.com/bevyengine/bevy/blob/main/crates/bevy_remote/src/http.rs
move cert gen to LY?
implement the ConnectionRequestHandler properly to reject invalid connections
move matchmaker bevy/http bits into matchmaker crate too? flag to run bemw server and mm service in one binary, using a thread for mm?
proto id and ly pkey needed in docker-compose for MM stuff too, perhaps a move to a standard lightyear.env
file which lightyear itself reads too? could default to sane zeros defaults if missing, for dev.
- env vars limited to 255 bytes. hacky util script added to pass ca_contents as docker runtime arg
- can't use
latest
tag for container images, it's cached aggressively.