-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Specify the exception for read failures arising due to partial data loss on the file system #423
Comments
It is important to note that a failure to read data stored on the disk can happen due to transient, recoverable reasons as well (low memory, lack of file descriptors, etc.). We can decide whether to specify an error for these scenarios too or to leave it to the implementer, but we should intend to distinguish between the recoverable and unrecoverable cases so that developers can handle the two cases appropriately. More details here - https://chromestatus.com/feature/5140210640486400 |
According to the WebIDL spec It was pointed out elsewhere that I'd propose pulling |
I agree that it's worth standardizing the error for cases where it's reported, and That said, it's been our belief in Firefox that unless we spec some kind of content opt-in to avoid the behavior, data corruption should result in the containing storage bucket being cleared. Multiple storage buckets seems like the most practical approach since it normalizes a hopefully uncommon case (data corruption) into a common case that code should already be handling (data clearing due to storage pressure). Related discussions: Storage corruption reporting. |
Thanks, Evan, Andrew. Let me work on a spec PR to:
Thoughts? Should we specify the conditions for throwing a
We discussed this briefly for Chromium too. Yes, deleting the storage bucket entirely would leave things in as consistent a state as possible, but it is a destructive action whose impact could vary based on how the website is using IndexedDB. Hence, it seemed better to surface the error clearly to the website so that it could take the right corrective action for its use case. We could discuss this further, though, and also perhaps solicit developer feedback. |
This seems good, thank you. I don't have a strong opinion other than, if there aren't many cases of needing the verbiage to throw the error and the verbiage isn't too verbose, having them in-line as opposed to in an extracted algorithm makes it more likely people will see that that can happen?
Yes, it would be good to discuss this further, and it need not be on this issue. I agree with the rationale that if a site can take corrective action, it would be good to enable the site to take corrective action. Unfortunately we've found many users of IDB don't really handle errors, so it would be desirable to have some kind of strong signal that the site is taking responsibility for dealing with the error. But I suppose that could be as simple as "the default behavior of a NotReadableError is clearing your bucket, so call preventDefault if you think you can clean things up". A meta issue for us is that right now, for non-Private Browsing Mode (PBM), we really only have SQLite's concept of detecting corruption and structured deserialization errors from JS, so if corruption happens, it's very hard to tell what still might be good. For PBM we actually use AEAD encryption on a database page level and blob chunk basis so we potentially have a much better idea of what is good and what is bad, and we might potentially move to using AEAD for non-PBM cases to that end in order to be able to better characterize the integrity of data. |
@asutherland, I've created #431 to discuss further on whether we should spec deletion of site data on corruption. |
IndexedDB data stored on the file system can be lost due to factors beyond the browser's control, such as when users directly delete the files containing the data.
Total data loss of the
database
orobject store
is visible to the script as an absence of data, and can be appropriately handled by it.However, since the current specification does not mandate consistent data storage, there is a chance of partial data loss where the entry for a
record
exists in theobject store
but the contents of therecord
are lost.For example, both Chromium and Firefox store blobs and large values in files separate from the main database, which only contains references to these files.
When those files are deleted, the references in the database become dangling and attempts to read the data fail permanently.
The specification does not cover the above scenario, and hence the
DOMException
thrown varies by browser:name
:NotFoundError
(updated recently)message
: "Failed to read large IndexedDB value"name
:UnknownError
message
: "The operation failed for reasons unrelated to the database itself and not covered by any other error code."Specifying a common
DOMException
for such cases of partial data loss will help developers handle these scenarios consistently across browsers.The text was updated successfully, but these errors were encountered: