-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Live subscription using Cosmos change feed #4
Comments
Hi @johnvanloon, thank you for asking - I was actually thinking about adding this feature, but decided to wait whether anyone really needs it. I'd like to keep the same API for both, but believe we can have this feature. We can have basically two approaches:
First approach is not perfect, but I think it is doable and could be handy to have the same functionality for both - we just need to come up with some reasonable mechanism for firing events ( Please, let me know what do you think about it. |
Agree that option 1 should be possible and also prefer that one over option 2. |
That would be perfect, thanks for that! I discussed with my friend @jbrestan skilled in pub/sub concepts how to do it for TableStorage so I'll update you with our progress. |
Sorry, took a little longer to get started. Thinking how I want to continue now. You got any great ideas? |
Ok. Forget that last comment. You are using SQL API. Somehow I assumed Mongo-API and somehow all tests also pass with a Mongo-API CosmosDb. Anyway, no blocking issues now... |
Hi John, I have new draft version in progress but need to write some tests. I think good way would be to expose IObservable (with observeOn thread pool) so consumers will not interfere with each other and can use reactive approach for hooking to newly appended events. The way I have it locally implement, the change feed is not actually needed. What do you think about it? |
Then how do you get the changes? Is it some kind of long-polling? |
No, it is internally normal .net event fired on each success append function. But with System.Reactive we can make it safe for consumers (and Rx approach would be more natural) |
But that only works if you're projection runs in the same process as the writer, right? I'm trying to trigger something in another service. |
Yeah, you are right. It can be consumed as any other .net event. I think you aim more for some kind of queue (Kafka, RabbitMQ), if you want to observe new events from different process. |
I just published v1.4.0 with |
Rethinking what I try to accomplish I agree that it seems to make sense to put it in a separate project. Basically what I'm trying to accomplish is an architecture as outlined in this blog series . Using that architecture, you use the event-store both for storage and for communicating with other services. You make the events the single source of truth for all services. For now I'll try to build things out in my own project. Maybe our paths will cross again in future. Thanks for your effort and discussions. |
I thank you! Good luck and let me know how it goes with your project. |
Was planning on building an eventstore for CosmosDb like this one.
One requirement would be to support live subscriptions, which could be implemented using the change feed of CosmosDb.
That would break the idea of having 1 interface for both storage implementations (or you would need to implement some kind of polling for Table Store or use the event grid notifications).
I'm totally fine to implement this myself, but I'm in doubt whether to create a PR as improvement for your project or fork/start over with a Cosmos-only implementation.
What would you prefer?
The text was updated successfully, but these errors were encountered: