SQLite DBs may be getting backed up to iCloud

[sithwarrior]

We are experiencing some issues where the database seems to be "out of sync" with the application state.

We need to be very offline capable, so images for content, and images taken by the user is stored in Library/NoCloud , and we manage what images are locally available in the sqlite db.

This works nicely, but we have been experiencing a number of users, throwing errors, that some images where not found. When investigating further, it turns out that the missing images, all were downloaded on "older" application folders*. We can see this, because we store the original full path of the images, in the sqliteDB.

example:
we are missing this image
var/mobile/Containers/Data/Application/EDFC715A-3FE0-457E-B9C3-2618E6F3F56B/Library/NoCloud/missing-image.jpg

EDFC715A-3FE0-457E-B9C3-2618E6F3F56B Being the current application folder.

However when going through all images in the DB, we can see that all images of the "oldest" application folders are missing.

Images from these are missing:

F2FBEB6D-33B5-467C-87A1-CB2AEFC83EF6
A917C3CF-6ECA-4FCE-AB1B-E80EBE1D776B

Images from these are all there:

A8D8070E-9E83-4EA4-895C-0EFD9EC09E40
F2146CFD-286D-423F-976E-BAE24C32AA64
2F553D81-AF68-4F78-ADA1-AD7328037AF9

Content is copied between app updates, so this shouldn't be happening, So what we think might be happening, is that in some circumstances the sqliteDB is being backed up to iCloud, and survives reinstalls, but the images obviously do not. This creates a very unfortunate missmatch.

Some more data:
This only happens on iOS
This happens in a single percentage of users

Im not any good at Objective C, but this part should be setting the Library/LocalDatabase to not sync

if ([[NSFileManager defaultManager] fileExistsAtPath: nosync])
        {
            DLog(@"no cloud sync at path: %@", nosync);
            [appDBPaths setObject: nosync forKey:@"nosync"];
        }
        else
        {
            if ([[NSFileManager defaultManager] createDirectoryAtPath: nosync withIntermediateDirectories:NO attributes: nil error:&err])
            {
                NSURL *nosyncURL = [ NSURL fileURLWithPath: nosync];
                if (![nosyncURL setResourceValue: [NSNumber numberWithBool: YES] forKey: NSURLIsExcludedFromBackupKey error: &err])
                {
                    DLog(@"IGNORED: error setting nobackup flag in LocalDatabase directory: %@", err);
                }
                DLog(@"no cloud sync at path: %@", nosync);
                [appDBPaths setObject: nosync forKey:@"nosync"];
            }
            else
            {
                // fallback:
                DLog(@"WARNING: error adding LocalDatabase directory: %@", err);
                [appDBPaths setObject: libs forKey:@"nosync"];
            }
        }

Which is great, but could something be happening, so this part fails?

While writing this, I was thinking maybe there could be a race condition, if open is called before pluginInitialize has finished?

Again objective C is not my forte, so any feedback would be appreciated.

[brodycj]

Thanks for the level of detail that you have given in the description.

I would be pretty surprised if open would be called before pluginInitialize has finished since Cordova initializes all plugins before sending the deviceready event and enabling the native plugin access.

The following technical note indicates to me that the code to set Library/LocalDatabase to not sync is correct: https://developer.apple.com/library/archive/qa/qa1719/_index.html

The one thing I can think of is that the error handling is not so good:

• one error is ignored
• there is a fallback to using Library

I do not expect these errors to occur in practice but think the code should be fixed to be 100% safe.

My one other idea would be to support an option to iosDatabaseLocation to store databases in Library/NoCloud instead.

---

P.S. To quote from https://developer.apple.com/library/archive/qa/qa1719/_index.html (copied directly from the HTML source):

Starting in iOS 5.1, apps can use either NSURLIsExcludedFromBackupKey or kCFURLIsExcludedFromBackupKey file system properties to exclude files and directories from backups.

and then the next 2 sentences:

Apps that need to exclude a large number of files can exclude them by creating their own sub-directory and marking that directory as excluded. Apps should create their own directories for exclusion, rather than excluding the system defined directories. Either of these APIs is preferred over the older, deprecated approach of directly setting an extended attribute. All apps running on iOS 5.1 and later should use these APIs to exclude data from backups.

I think this confirms that I did already get the code right.

[sithwarrior]

Thanks for the reply Brody

Im also seeing it as Library/LocalDatabase is set to not sync, as per Apples own docs.

If im reading the opennow method correcltly, the fallback dblocation is actually Documents ?

NSString *dblocation = [options objectForKey:@"dblocation"];
if (dblocation == NULL) dblocation = @"docs";

Would be nice to have the option to store the db in Library/NoCloud , but as we are using localForage, and the localForage sqlite driver on top of this plugin, there is no easy way to make use of such an option, or am I missing something?

To debug this further I have created some new logging that should give me the contents of:

• Library/NoCloud
• Documents
• Library/LocalDatabase

I hope this will give a better understanding of what might be happening.

[brodycj]

Thanks again @sithwarrior for reporting this. I just pushed an update to be hopefully extra-safe against this kind of thing happening due to an uncaught error. I will very likely publish a new major release later today or tonight. (Major since it looks like a potentially breaking change.)