question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow consumption of Change Feed as a pull model

See original GitHub issue

Is your feature request related to a problem? Please describe. We are using the ChangeFeedProcessorBuilder in our WebAPI to update an in-memory cache with recent changes to a container as outlined in the change feed use cases documentation. We have multiple instances of our WebAPI that each have their own cache so each instance is working on a single processor and doesn’t need to share updates.

Describe the solution you’d like Allow us to bypass the need for a lease container to read from the change feed. One solution might be to make ChangeFeedProcessorBuilder.WithInMemoryLeaseContainer() public with a way of specifying a starting continuation Token if desired.

Another would be to expose a container method that allows us to make a direct request for the change feed with our own continuation token and leave it to the caller to write the timer interval and storage mechanism (if desired).

Describe alternatives you’ve considered Giving a unique processor name per WebAPI does work but it is unnecessarily creating lease documents and updating them often (we are seeing ~22 requests per minute on our lease container per WebAPI. This multiplies based on number of logical partitions). WithLeaseConfiguration() seems to allow overriding the defaults to be much higher but there is no guidance for what appropriate values would be.

Additional context Failing to provide a WithLeaseContainer() call on the ChangeFeedProcessorBuilder results in an exception saying to add it or use WithInMemoryLeaseContainer which is not accessible.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:16 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
ealsurcommented, Feb 13, 2020

@WimVergouwe please see linked PR

1reaction
ealsurcommented, Nov 4, 2019

In-memory would work like the current CFP, either start from the beginning, or start at some point in time, or from now on. No way to pass any other state.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Change feed pull model in Azure Cosmos DB
To process the change feed by using the pull model, create an instance of FeedIterator . When you initially create FeedIterator , you...
Read more >
Reading Azure Cosmos DB change feed
The change feed pull model allows you to consume the change feed at your own pace. Changes must be requested by the client...
Read more >
First steps of the Change Feed pull model in Azure ...
The beauty of Change Feed is that it can read data in the order in which it is updated, as long as the...
Read more >
First steps of the Change Feed pull model in Azure Cosmos DB
The pull model is a very thin API compared to the Change Feed Processor, but allows developers to have more control and flexibility....
Read more >
Azure Cosmos DB change feed
If you need to control change feed consumption at a customized pace, use the pull model. For more information, see Change feed pull...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found