Change feed pull model - High request charges
See original GitHub issueDescribe the bug Hi, I’m not sure if this is a bug or expected behavior but here it is.
I’m trying to use the change feed pull model to query all documents in a specific partition key and save the continuation token when there is nothing left to read. In the next run, I will be able to use the saved continuation token to continue querying the change feed for new changes. The thing is, I’m not sure if the request charge should be that high. For only 141 documents in the collection, I see request charges from 10 000 to almost 25 000. See below.
var changeFeedIterator = _eventStoreContainer.GetChangeFeedIterator<EventStoreEvent<object>>(
ChangeFeedStartFrom.Beginning(FeedRange.FromPartitionKey(new PartitionKey(instance.Id))),
ChangeFeedMode.Incremental
);
while (changeFeedIterator.HasMoreResults)
{
var response = await changeFeedIterator.ReadNextAsync();
if (response.StatusCode == HttpStatusCode.NotModified)
{
Console.WriteLine($"No new changes");
break;
}
eventsToReplay.AddRange(response);
}
I tested with a simple LINQ query and the request charge was only 11.39 to query the same 141 documents
var feedIterator = _eventStoreContainer.GetItemLinqQueryable<EventStoreEvent<object>>(requestOptions: requestOptions)
.ToFeedIterator();
while (feedIterator.HasMoreResults)
{
var response = await feedIterator.ReadNextAsync();
eventsToReplay.AddRange(response);
}
So is this normal? I’m expecting a higher request charge because it needs to read all the changes in the partition but should it be that high?
Environment summary .net core 3.1 “Microsoft.Azure.Cosmos” Version=“3.24.0” Windows 11
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (2 by maintainers)
Top GitHub Comments
Here are the diagnostics logs diagnostics.zip It contains the logs for the change feed iterator call and the LINQ queryable call Thank you
Thank you for the information.
The thing with query ORDER by _ts is I cannot continue where I was on the next function run as I do with the ChangeFeedIterator continuation token. The idea was to use the ChangeFeedIterator with a continuation token to have an on-demand event processing from an event store. I cannot use the _ts as multiple events can have the same _ts and we don’t want to process the same event twice.
Is there an estimated date/planned fix for this?