Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial ARCHITECTURE document #749

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Architecture

Rojo is a rather large project with a bunch of moving parts. While it's not too complicated in practice, it tends to be overwhelming because it's a fair bit of Rust and not very clear where to begin.

This document is a "what the heck is going on" level view of Rojo and the codebase that's written to make it more reasonable to jump into something. It won't go too into depth on *how* something is done, but it will go into depth on *what* is being done.

## Overarching

Rojo is divided into two main pieces: the server and the plugin. The server is what's ran on your computer (whether it be via the terminal or the visual studio code extension), with the plugin serving as its main client.
Dekkonot marked this conversation as resolved.
Show resolved Hide resolved

When serving a project, the server gathers data on all of the files in that project, puts it into a nice format, and then sends it to the plugin. Then, when something changes on the file system, it does the same thing for only the changed files and sends them to the plugin.

When it receives a patch (whether it be the initial patch or any subsequent ones), the plugin reads through it and attempts to to apply it. Any sugar (the patch visualizer, as an example) happens on top of the patches received from the server.
Dekkonot marked this conversation as resolved.
Show resolved Hide resolved

## Server

Rojo's server component is divided into a few distinct pieces:

- The web server
- The CLI
- The snapshotting system

### The CLI

The Command Line Interface (CLI) of Rojo is the only interface for the program. It's initialized in `main.rs` but is hosted in `src/cli`.

Each command for the CLI is hosted in its own file, with the `mod.rs` file for the `cli` module handling parsing and running each command. The commands are mostly self-contained, though may also interface with Rojo's other code when necessary.

Specifically, they may interface with the web server and snapshotting system.

### The Snapshotting System

To do what it does, Rojo has to do two main things: it must decide how the file system should map to Roblox and then send changes from the file system to the plugin. To accomplish this, Rojo uses what's referred to as snapshots.

Snapshots are essentially a capture of what a given Instance tree looks like at a given time. Once an initial snapshot is computed and sent to the plugin, any changes to the file system can be turned into a snapshot and compared directly against the previous snapshot, which Rojo can then use to make a set of patches that have to be applied by the plugin.

These patches represent changes, additions, and removals to the Roblox tree that Rojo creates and manages.

When generating snapshots, files are 'transformed' into Roblox objects through what's referred to as the `snapshot middleware`. As an example, this middleware takes files named `init.lua` and transforms them into a `ModuleScript` bearing the name of the parent folder. It's also responsible for things like JSON models and `.rbxm`/`.rbxmx` models being turned into snapshottable trees.

Inquiring minds should look at `snapshot/mod.rs` and `snapshot_middleware` for a more detailed explanation.

Because snapshots are designed to be translated into Instances anyway, this system is also used by the `build` command to turn a Rojo project into a complete file. The backend for serializing a snapshot into a file is provided by `rbx-dom`, which is a different project.

### The Web Server

Rojo uses a small web server to forward changes to the plugin. Once a patch is computed by the snapshot system, it's made available via the server's API. Then, the plugin requests regularly, and if a new patch exists, recieves it and applies it in Studio.
Dekkonot marked this conversation as resolved.
Show resolved Hide resolved

The web server itself is very basic, consisting of around half a dozen endpoints. The bulk of the work is performed by either the snapshot system or the plugin, with the web server acting as a middleman.

## The Plugin

This section of the document is left incomplete.
boatbomber marked this conversation as resolved.
Show resolved Hide resolved
boatbomber marked this conversation as resolved.
Show resolved Hide resolved

## Data Structures

Rojo has many data structures and their purpose might not be immediately clear at a glance. To alleviate this, they are documented below.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also document the structures in the plugin, like PatchSet. It's particularly important because of how inconsistent things are with camelCase and PascalCase in our structures across the project.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to do this with Plugin and Server headers in this section due to the duplicate naming (ie: ServeSession) making the distinctions critical.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up
Because this doc is just meant as a high level introduction to Rojo functionality, we don't need to document the details of every plugin structure. I do think PatchSet is still very important to document though, because its the main thing the plugin interacts with during core functionality.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a dedicated Plugin subsection that this can go in if need be, though maybe it's worth giving the Plugin its own document?


### Vfs

To learn more, read about [`memofs` architecture](crates/memofs/ARCHITECTURE.md).

### ServeSession

### ChangeProcessor

### RojoTree

### LifeServer

### InstanceSnapshot
48 changes: 48 additions & 0 deletions crates/memofs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Architecture

As the `README.md` says, this is an incomplete library that's dedicated to serving Rojo's purposes for the time being.
Meaning, there's still plenty of work to be done and the API will need to change.

For the time being, this documents the current state of affairs.

## VFS Interface

The predominant object of `memofs`, it provides an interface for an abstract filesystem (known as a backend in `memofs`).

### Backends

Instead of `Vfs` providing a backend for you, it's flipped that it consumes a backend to use.
Essentially, [late-binding](https://ericlippert.com/2012/02/06/what-is-late-binding/) the backend so it's decided at runtime instead of compile-time.

This is useful because you can implement any arbitrary backend and pass it along to `Vfs` without any changes to your project!

For example, if you want to use a network drive instead of a local file system,
all that's required is to implement a `VfsBackend` to interface with the network drive.
Now, you can swap your local file system for the new one!

There are common use cases for this feature, hence `memofs` provides several backends.

#### In-memory

As the name implies, it keeps all files and directories in memory.
This is particularly useful for testing, as it's easy to build, snapshot, and teardown.

To help, `memofs` provides a `VfsSnapshot` object to snapshot the filesystem. The in-memory backend has methods to load from and save to a `VfsSnapshot`.

#### Noop

As the name implies, it does nothing.
As the name doesn't imply, every operation will error.
This is useful if you want to verify that your software doesn't perform any read/write operations.

#### Std

As the name implies, it provides an interface to the `std`'s filesystem API. Particularly, `fs_err` for nicer error messages!

### Filesystem events

`Vfs` additionally provides an event bus via `Vfs::event_receiver()`.
For any changes detected by the backend, it will be sent down that channel.
Only `std` actually provides any events.

Additionally, there is a `Vfs::commit_event()` method, which will unwatch a path if a remove event is passed.