Skip to content

Commit

Permalink
Split the architecture into 3 categories.
Browse files Browse the repository at this point in the history
The architecture is now:
* components
* deployment
* flows
  • Loading branch information
dopiera committed Sep 20, 2024
1 parent cb20216 commit 1addfcb
Showing 1 changed file with 63 additions and 68 deletions.
131 changes: 63 additions & 68 deletions arch-snapshot/arch.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,55 +110,37 @@ This word is used to describe Offer/Demand put on market, so we should mention i
### Offline requestors are not supported
### Local storage (TODO: what role does the local DB play?)

## Technical view - architecture layers
Bottom-up description of different layers of Golem that impose structure on how
we split components and facilitate communication between them. Golem is not
monolithic and division of responsibility is not arbitrary.

The goal of this chapter is to sketch general idea, rather than describe
specifics.
### Network layer
### GSB layer
RPC-like communication layer providing a way to call other modules or other
nodes in unified way.
### Specialized modules layer (market, payment, activity, vpn etc.)
#### Market negotiation protocols
Offers/Demands property and constraint language is used to build subprotocols
for negotiating certain aspects of Agreements.
### Execution environments
There are a few ways to implement an execution environment. We have a few APIs
here:
* Bind and handle GSB messages directly
* Implement RuntimeSDK and use ExeUnit binary for common functionalities
* Command specification which can be executed by ExeUnit is flexible enough that
higher level protocols could be created here.
### API layer
Each specialized module exposes some APIs to users. It is either REST API, yagna
cli or set of GSB endpoints for communication between modules and nodes.
#### REST API
#### CLI
#### GSB API (communication between modules)
### SDKs layer (python and js SDK)
#### Agent application (Requestor script, Provider Agent)
### Higher level SDKs
#### Golem compose
#### Ray on Golem

## Technical view - functional aspects
This section dives into aspects of Golem network architecture to explain how
they work under the hood. The level of detail does not include code structure,
interfaces nor data structures but names key components and interaction between
them.

### Market interactions
#### Discovery and Offers/Demand matching
- Offer/Demand properties and constraint langauge
- Yagna is property agnostic - doesn't understand semantic of properties, only agent do
- Some examples of properties and costraints and how it works
- Links to more detailed docs for properties langauge and properties sepcification (?)
#### Offer propagation
- Link to design [decision](#only-providers-offers-are-propagated)
- Algorithm overview
## Technical view - components
This section describes key components of Golem Network, i.e. their
responsibilities, interfaces and which other components they utilize.

### Networking
* how it works that two separate Yagnas can talk to each other
#### Central net
#### Hybrid net
- Identification
- Relay
- Discovering Nodes
- P2P communication
- Relayed communication
- Cryptography
- Node identity verification (challenges)
- Communication encryption

### GSB
* what it is, how it works and how it imposes a code structure and how
addressing works

### Offer / negotiation
A description of the component responsible for making offers, counter-offers,
negotiations, etc.

* Offer/Demand properties and constraint langauge
* Yagna is property agnostic - doesn't understand semantic of properties, only
agent do
* Some examples of properties and costraints and how it works
* Links to more detailed docs for properties langauge and properties
specification (?)
#### Process of negotiations and making an agreement
- Initial Proposal
- Countering Proposal
Expand All @@ -173,22 +155,14 @@ them.
- What is specified by protocol and what is left to future specifications?
- Termination reason concept

### Communication between nodes
#### Central net
#### Hybrid net
- Identification
- Relay
- Discovering Nodes
- P2P communication
- Relayed communication
- Cryptography
- Node identity verification (challenges)
- Communication encryption
#### GSB communication through Net
- Communication is independent from net implementation
- GSB addressing
### Offer propagation
* a description of how it happens that offers are visible to reqestors
* Link to design [decision](#only-providers-offers-are-propagated)
* Algorithm overview

### Payments & Clearing
### Payments
* a description of current payment driver, its modes of operations and how it
can be extended
#### Payments models
- Describe generic model which is open for new implementations
- Payment model specification in Offer/Demand language
Expand All @@ -210,7 +184,10 @@ them.
- Overview of the concept
- Link to external documentation describing details

### ExeUnits
### Activity
* How the actions on behalf of the requestor are performed
* We should dive into each important and general implementation, i.e. WASM and
VM
#### Abstract concept
- ExeUnit concept is generic enough to sell any kind of computation resources
- Generic ExeUnits (for example VM, WASM etc.) vs. specialized ExeUnits for specific tasks like:
Expand Down Expand Up @@ -247,10 +224,28 @@ them.
- WASM images

### VPN
### Processes and the responsibility split between them
#### IPC
* The component responsible for creating a VPN between VMs

### Reputation
* a description of how it is evaluated, distributed and used

### SDK
* which of the logic useful to the user ends up in the SDK

## Technical view - deployment
How the components are reflected in processes, where the processes are run, what
is their relation ship, etc.

## Technical view - flows & algorithms
This section documents how control and responsibility flows through the listed
components to achieve Golem's functionalities. Any non-trivial algorithms
spanning more than one component are also described here.

### Reputation management
### Starting a provider and publishing an offer
### Receiving and executing work
### Finding a provider and requesting work
### Starting a cluster of VMs
### Creating a custom image

## Key architectural shortcomings
This section contains known shortcomings of the implemented architecture —
Expand Down

0 comments on commit 1addfcb

Please sign in to comment.