Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync Adapter #341

Merged
merged 26 commits into from
Feb 22, 2018
Merged

Sync Adapter #341

merged 26 commits into from
Feb 22, 2018

Conversation

jnunemaker
Copy link
Collaborator

Adds a sync adapter. When you have a remote that is relatively latent (say http and not in same datacenter, region or whatever) and want a local "cache" that is more persistent than a cache (across process restarts and serves stale instead of raising if remote is unavailable), this adapter can come in handy.

  • Reads go to local adapter (so reads are fast).
  • Writes go to remote adapter and then to local adapter if remote successful.
  • Every N milliseconds, the local adapter will be diff'd against the remote and appropriate enable/disable calls will attempt to converge them.
  • All syncing is done inline (nothing fancy in a background thread at this point).
  • Any errors during sync are swallowed (but instrumented for visibility) to avoid causing any disturbances to the request/job currently being performed.

Long way to go yet. I still want this to:

* sync from remote if local is blank
* sync local from remote on interval
This is naive as it happens in the synchronous path instead of in a
background thread but is ok to start.
This is to prevent the remote being unavailable from affecting the app
other than a timeout every interval.
Need to get some units for each of them as well.
Also now instrumenting call so people can keep track of how often the
synchronizer is invoking and how long it is taking.
Running into some weird stuff and I'm wondering if it is struct. Some
kind of endless loop for both "p" and "pp" for operations.
@jnunemaker jnunemaker self-assigned this Feb 22, 2018
@jnunemaker
Copy link
Collaborator Author

I'm going to keep moving forward, but would love it if anyone would give this a look over.

@jnunemaker jnunemaker merged commit d9499be into master Feb 22, 2018
@jnunemaker jnunemaker deleted the sync-adapter branch February 22, 2018 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant