Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update data models #127

Open
sgdavis1 opened this issue Aug 2, 2022 · 3 comments
Open

Update data models #127

sgdavis1 opened this issue Aug 2, 2022 · 3 comments
Assignees

Comments

@sgdavis1
Copy link
Member

sgdavis1 commented Aug 2, 2022

In the research and planning for #111 and #93, we have identified that there is an unnecessary redundancy between the database data models and the memory models used to manage the backup system in lib/system.js:machines:17 and lib/system.js:prepareMachines():145-164. The loose typing of JS has caused the problems mentioned above because of inconsistencies in the way the system.js reads and creates this models at init() and the way they are created when a backup is completed during active processing by the rsync.js module. They are also lightly documented and hard to determine expectations for the codebase because this documentation is not centralized anywhere.

As a solution we will rethink our database and move all of these memory models into the database. This process will require a codebase reorganization but will result in a much cleaner and more maintainable system. Use this issue to describe the plan for this work.

@sgdavis1
Copy link
Member Author

sgdavis1 commented Aug 2, 2022

Current structure:

backupEvent: {
    'machine': DataTypes.STRING,
    'schedule' : DataTypes.STRING,
    'bucket' : DataTypes.STRING,  // Not sure what this is
    'rsyncExitCode' : DataTypes.INTEGER,
    'rsyncExitReason' : DataTypes.STRING,
    'transferSize' : DataTypes.INTEGER,
    'transferTimeSec' : DataTypes.INTEGER,
    'dir' : DataTypes.STRING,
    'backupTime': (timestamp)
}

sizes: {
    'machine' : DataTypes.STRING,
    'location' : DataTypes.STRING,
    'size' : DataTypes.INTEGER
}

No current foreign key relationships exist. We will at least need a machine object added to this and both of these will be adjusted. Suggest the new schema below.

@Jacks-01
Copy link

Jacks-01 commented Aug 3, 2022

A couple questions,

No current foreign key relationships exist. We will at least need a machine object added to this and both of these will be adjusted. Suggest the new schema below.

so the machine property exists in sizes.js, backupEvents.js, and system.js. When you say you want a new machine object, are you implying we should create a new machine model and scrap the old ones? It's somewhat confusing because machine is referred to in many different places.

On a side note, would creating an ER diagram be helpful for documentation / clarification?

@ashleyseo
Copy link

Here is what we were thinking about so far, do you have any suggestions?

`backupEvent: {
'machine': DataTypes.STRING,
'schedule' : DataTypes.STRING,
'rsyncExitCode' : DataTypes.INTEGER,
'rsyncExitReason' : DataTypes.STRING,
'transferSize' : DataTypes.INTEGER,
'transferTimeSec' : DataTypes.INTEGER,
'dir' : DataTypes.STRING,
'backupTime': DataTypes.STRING
}

sizes: {
'machine' : DataTypes.STRING,
'location' : DataTypes.STRING,
'size' : DataTypes.INTEGER
}

machine : {
'schedule' : DataTypes.STRING,
'name': DataTypes.STRING,
'host': DataTypes.STRING,
'port': DataTypes.INTEGER,
'backupDirectories': DataTypes.ARRAY,
'lastBackup': DataTypes.STRING,
'totalSize': DataTypes.INTEGER
}`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants