Migration Roadmap

[nathan-isaac]

We may want to break some of these into other discussions but for now I'm going to consolidate them all here.

Contributions

I updated the readme to include some high level architecture plans.

Routes to migrate

• /* Static Assets

• /account

  • [GET] /needs-bootstrap
  • [GET] /validate
  • [POST] /bootstrap
  • [POST] /login
  • [POST] /change-password

• /sync

  • [POST] /sync
  • [POST] /user-get-key
  • [POST] /user-create-key
  • [POST] /reset-user-file
  • [POST] /upload-user-file
  • [POST] /download-user-file
  • [POST] /update-user-filename
  • [POST] /delete-user-file
  • [GET] /list-user-files
  • [GET] /get-user-file-info

CRDT

• Migrate merkle.js
• Migrate timestamp.js

Protobuf

• Migrate protobuf logic - https://github.com/protocolbuffers/protobuf-go

Other tasks

• Automate test on push
• Automate releases with https://goreleaser.com/

[irfanbacker]

Hey, SantaClaus from discord here.The proposed architecture looks great! But I'm not exactly sure if assigning tasks according to the routes is ideal at this initial stage. All of these routes contain underlying module dependencies which overlap. I think implementing these modules could be done initially and then the routes could be migrated individually.

The modules could be seperated as:

• Timestamp and Merkle (timestamp.js and merkle.js from loot-core/src/server)
• SyncProtobuf
• Database

What do you think?

[irfanbacker]

The account routes are self-contained but depend upon the database setup.

The syncing routes depend upon both Database as well as the core CRDT logic. As you suggested, creating internal.core.crdt for Timestamp and Merkle logic seems to be the right way to go. We could use https://github.com/roberson-io/mmh3 and https://github.com/google/uuid as dependencies for Timestamp. The Merkle logic won't need any external dependencies.

The package you mentioned for protobuf is deprecated, I believe. We could use protobuf-go to generate it. The .proto file is located here.

So, what I meant earlier was that, there are 3 self-contained tasks that we could assign ourselves right now. We could proceed to the other tasks after completing these 3 initiallly. All the other leftover tasks will be self-contained after this.

[irfanbacker]

Hey @nathanjisaac . I have implemented the basic database functions in my fork. Do take a look at it. I have implemented every function for WrappedDatabase except for transactions. The API in golang is very different from Javascript for this. So I think it'll be easier to use the default API from database/sql like db.Begin().

Let me know if any changes are needed. I'll create a PR for this if the changes are fine.

[nathan-isaac]

Awesome. Thanks. I'll take a look.

[irfanbacker]

I have generated the protobuf from sync.proto. I have added a readme on how to regenerate it, if needed. Also, I have created PRs for both of these. Ig that'll make it easier for reviewing.

[nathan-isaac]

Awesome. Planning to look at this later this afternoon.