Email structures
The system internally should be pluggable. We have a single interface in which we can implement a whole file storage system. This means it would be fairly simple to change from a file-system to a database-storage system (or hybrid) instead.
.maildb/
1d/ # first two chars of the sha256 hash of the email address
9A6E493390DA308F5A082E4DC4ABA11BC0B4C0829BB3DCE222897E8B3CCE3A/ # Remainder of the sha256 hash of the email address
.keys.json # Public keys for this account.
box-1/ # mail directories
9fe27f14-a30b-46c9-9303-060dbcbae92d/ # directory per email, consisting of email ID
header.json # header file of email (you need to have key for decoding catalog.json.enc)
catalog.json # encrypted catalog.json file (see below)
34e941f2-5e5f-4855-94a1-73572a5f8f29.enc # one or more blocks or files, based on ID from the catalog (encrypted, but not base64 encoded)
113c9c21-3319-4026-92d2-57181734410a.enc
504a2dfb-1231-4714-a5de-df6d184fbddc.enc
box-2/
box-3/
box-100/
...
The message header file is an unencrypted file that allows mail servers to deal with incoming emails. It should contain as little metadata as possible.
{
"from": {
"address": "1db14ca62f80ce8c77ae749ddf04e2d4077451db9f701f3bd37dec1dc950c76b"
"public_key": "rsa MIICCgK....wuLz8koxHFpbFEmMecqTbUdweNi7MeerYK07HG6L7MB9y/HzJ7Ig9eYvZXcErGh3R5fDq7aJSdp9arHbuL0PyNti5qoUrUBC5mCVdcvlg+7V19sX"
"proof_of_work": "22$data$396056"
},
"to": {
"address": "19b14ca62f80ce8c77ae749ddf04e2d4077451db9f701f3bd37dec1dc950c76b",
},
"catalog": {
"size": 1728,
"checksum" : [
{ "hash": "sha1", "value": "8b53323e62705215db067ad8f296f490a57b89cf" },
{ "hash": "sha256", "value": "61b0476f8e249c68bbaa14222de9f29f078133ff9d0b461d69b78f5a63f5b678" },
],
"txid": "yB665L6BtKlFNI38Vg6jUaJ3SXYi0R7vGUmAfWBiuaI5G2nBw0i1IDZAu3dVundBE3PJsU48eyAt1JAEQ7I0r"
"encrypted_key": "hlyhaNEVfLfkHVCcexyMXGLRTGPaZElrj49ktvFF..../Szt7+Bu7tJWRVno"
},
"sender_signature": "....."
}
from.address The hashed email address that sends the message
from.public_key The public key of this user (TODO: do we need this?)
from.proof_of_work Proof of work for the user account
to.address The hash of the address that receives the email. This email box should exist on the server
catalog.size Size (in bytes) of the catalog file
catalog.checksum[].hash The hash method for the given checksum
catalog.checksum[].value Actual hash value for the whole catalog file
catalog.txid The transaction ID
catalog.encrypted_key Encrypted key to decrypt the catalog. Only the receiver is capable of using this key.
sender_signature The signature of the sending mail server.
The catalog file consists of all message meta-data and the catalog of the blocks. This file is encrypted and cannot be read directly from the mail server.
{
"catalog": "RV5Ac2yroHISyHnS/+3vLF3n0OYtGpJ38YZdvovL3Jt8Yf1yrUR0V3STYjf4PxpDMXhvhPJXpeCw0YENpkTfsT/l8xIH4fTB2Cko5QNb",
"crypto": "rsa+aes256gcm"
"encrypted_key": "hlyhaNEVfLfkHVCcexyMXGLRTGPaZElrj49ktvFFZf8eo/wvUJaG/Szt7+Bu7tJWRVnoyP2kldKl79ZrxtfmQOcXw1spJO8pjrqxy1ZM68KnawOW2jPqjNBexJER4ibkRF2ILWZplo4bumrvpfuX80HGZXeFCO7MP/SdUtyvxt/BMfJv4E/l+q/6YfP+apMvyPiEkwg4CaBRXhXhyB665L6BtKlFNI38Vg6jUaJ3SXYi0R7vGUmAfWBiuaI5G2nBw0i1IDZAu3dVundBE3PJsU48eyAt1JAEQ7I0rh3D9VfzCojRGM9pYxcQfGO0/lu0zinl+WTpSfNsOuzvg2MsSKXq5VTMJnNQv4tIYoEY68rOwa4XR//2s8FdfB2K84vWiNhrMOvPz4QqQ04yLq/4VrRB6L881JRpXEt7l7bcLPuQDyGBl8kq+k5ONpMfs/w/+WjySd8N/PGPyXBhnKWP9YnxpHbqZ+t1AyNj9bYdcyOqzUHQLk3/jAq0z7VTInYiV/vTztR2ZdKewm2Yfl1r4uH3Z7d/v6W+Vgy/DuONvF433Z9vAfuoWbtBM+qqzZuFZH6DWSFWicZkFYF50pN1rooafmycM9yemWkaOdN8tE/u26puOacH9R87stSjjNbdy9CXWy+ZJ7Jjj62o3SzfmbFeOi/24Vd0NfvMDpZkDjo="
}
catalog Encrypted catalog data
crypto Crypto used (rsa+aes supported)
encrypted_key RSA encrypted 256bit key
{
"from": {
"address": "joshua!",
"name": "Joshua Thijssen",
},
"to": {
"address": "info@seams-cms!",
"name": "Seams-CMS"
},
"created_at": "2020-05-25T06:58:28+00:00",
"thread_id": "bb9d16ea-b232-4280-b1cd-aead2edf0c0c",
"subject": "Your invoice 2010-14141 from NoxLogic",
"labels": [
"invoice",
"service"
],
"catalog": {
"blocks": [
{
"id": "113c9c21-3319-4026-92d2-57181734410a",
"type": "html",
"size": 149,
"encoding": "base64",
"compression": "gzip",
"checksum": [
{
"hash": "sha1",
"value": "c1d68280163089ceff11c54d6162a9504d9330f3"
},
{
"hash": "sha256",
"value": "f3c4de175280e55a630b11563f1d9d4a037f6f8017a548037dc75df10deb4435"
},
{
"hash": "crc32",
"value": "d4a14a92"
}
]
},
{
"id": "34e941f2-5e5f-4855-94a1-73572a5f8f29",
"type": "mobile",
"size": 149,
"encoding": "base64",
"compression": "gzip",
"checksum": [
{
"hash": "sha1",
"value": "804beebfc9b8078a262be51edf2d358ab1b91d85"
},
{
"hash": "sha256",
"value": "a899261a63a0cd7e2386ec8a214707e5fd3b9364cc7e90109c774f80b422bb58"
},
{
"hash": "crc32",
"value": "efa3ba3e"
}
]
},
{
"id": "504a2dfb-1231-4714-a5de-df6d184fbddc",
"type": "text",
"size": 101,
"encoding": "base64",
"compression": "gzip",
"checksum": [
{
"hash": "sha1",
"value": "63f69cfa77e8c03d1ee401d701a0dae901f15e9a"
},
{
"hash": "sha256",
"value": "ca24bc0c7ac3964dda00eb66dcceba016c16a3d8e8a87f2294bbadef3aaecdf6"
},
{
"hash": "crc32",
"value": "54b62f2c"
}
]
}
],
"attachments": [
{
"id": "0f5f38cd-fdea-4dc0-a837-6d06f9c37a34",
"mimetype": "application/binary",
"filename": "textfile.doc",
"size": 14914141,
"encoding": "base64",
"compression": "gzip",
"checksum": [
{
"hash": "sha1",
"value": "c1d68280163089ceff11c54d6162a9504d9330f3"
},
{
"hash": "sha256",
"value": "f3c4de175280e55a630b11563f1d9d4a037f6f8017a548037dc75df10deb4435"
},
{
"hash": "crc32",
"value": "d4a14a92"
}
]
}
]
}
}
from.address Sender mail address
from.name Sender name
to.address Mail address of the receiver
to.name Name of the receiver
created_at ISO 8601 date when the message was created
thread_id Thread ID (if any). ThreadID.ParentID
subject Subject of the message
labels[] Additional labels (invoice, service, important etc)
flags[] Additional flags ()
catalog.blocks[].id UUID of the block
catalog.blocks[].type Block name (html, mobile, text)
catalog.blocks[].size Size of the block in bytes
catalog.blocks[].crypto Crypto used (rsa+aes256gcm supported)
catalog.blocks[].key RSA encrypted 256 bit key
catalog.blocks[].compression Compression method used (gzip)
catalog.blocks[].checksum[].hash The hash method for the given checksum
catalog.blocks[].checksum[].value Actual hash value for the whole catalog file
catalog.attachments[].id UUID of the file
catalog.attachments[].mimetype Mimetype of the file
catalog.attachments[].filename Original filename
catalog.attachments[].size Size of the block in bytes
catalog.attachments[].crypto Crypto used (rsa+aes256gcm supported)
catalog.attachments[].encrypted_key RSA encrypted 256 bit key
catalog.attachments[].compression Compression method used (gzip)
catalog.attachments[].checksum[].hash The hash method for the given checksum
catalog.attachments[].checksum[].value Actual hash value for the whole catalog file
