Local data storage and Excel upload/download #27
Conversation
…t for excel sheet
…t line items in the fund
|
@maxatdetroit If you have a chance, could you look at the way I'm handling and storing the Excel data and give me your thoughts? The relevant functions are in Basically, I'm reading in the Excel file, saving it in local storage in a JSON-like format, editing that copy with any user changes, and then using the same data to write back to an Excel file available for download at the end. Local storage doesn't seem perfect for this, and the initial Excel file read-in is a bit slow, but I think it might be the best (only?) way to do it with a pure front-end solution. I've been using the Excel file in |
| workbook.SheetNames.forEach(sheetName => { | ||
| // only convert sheets we need | ||
| if (Object.keys(SHEETS).includes(sheetName)) { | ||
| // read in sheets | ||
| const sheet = workbook.Sheets[sheetName]; | ||
| const rawData = XLSX.utils.sheet_to_json(sheet, { header: 1, defval: null }); | ||
| const rawData = XLSX.utils.sheet_to_json(sheet, { header: 1, defval: '' }); |
There was a problem hiding this comment.
the initial Excel file read-in is a bit slow
It's hard to say if it's the arraybuffer read / Blob write or JSON serialization / deserialization that is slowing things down without profiling the code. How bad is the performance? How does it scale with the size of the file / data?
Local storage doesn't seem perfect for this, ... , but I think it might be the best (only?) way to do it with a pure front-end solution
If the performance hit is in the JSON serialization / deserialization, then there is the option to use IndexedDB which is a client-side NoSQL style storage. Then we could store the data as a binary blob instead of JSON. That would have a performance improvement but require some extra effort to setup.
Is the performance hit worth addressing in your opinion?
There was a problem hiding this comment.
It's not that slow -- a couple seconds. Based on a quick look into IndexedDB, it's probably not worth changing the approach for now, but I'll test the current code to see how long it takes to process some of the larger departments.
There was a problem hiding this comment.
Yea, I agree. A couple seconds isn't bad. A better, lower-effort fix would probably be to handle the IO and serialization operations as asynchronous operations and provide some UX treatment like a spinner while they process.
js/views/view_logic.js
Outdated
| @@ -81,6 +81,6 @@ export function lastPage(){ | |||
| } | |||
|
|
|||
| export async function pauseAndContinue(){ | |||
There was a problem hiding this comment.
Why do we pause execution before proceeding in the app when the user clicks the next button?
There was a problem hiding this comment.
I was trying to smooth the transition between pages, but it doesn't really look as intended, so I'll delete.
Closes #23. Related to #20.