[Firestore] Question: Recommendations for FirestoreClient / FirestoreDb lifecycle management for backend services

Following @jskeet suggestion I have created separate issue.

Hi, I’m using Google.Cloud.Firestore package in version 1.0.0-beta22. What are current recommendations for lifecycle management of FirestoreDb / FirestoreClient especially for backend services?

I use Firestore in backend service to process telemetry data (intensive reads and writes are done there). Service is deployed in GCP GKE and I observe high latency there. I followed all best practices but still 99th percentile is over 200ms (for reads and writes). Such results were present when I had single FirestoreDb instance for each pod.

I achieved lower latency by using basic pool mechanism. Created specified number of FirestoreDbs and then fetch them in random way. All these FirebaseDbs are created using the same json credentials:

var client = new FirestoreClientBuilder { JsonCredentials = credentialsJson }.Build();
return FirestoreDb.Create(projectName, client);

Are there any other recommendations for backend services to achieve lower latency?

[jskeet]

Thanks for the extra detail. I probably won’t investigate this any further for now, but that will be useful if we ever come across this again.

Best of luck with your application, and I hope Bigtable works out well for you.

[pblachut]

Sorry for delay. We abandoned the idea of using Firestore in final solution, we have chosen to use Bigtable. Really appreciate your help and I think you have already answered my question.

I looked into the code which you have prepared and in comparision to ours there are few small differences.

1. we used different api to add to batch document:

• in our case: batch.Set(documentRef, document);
• in your repro: batch.Create(documentRef, document);

1. we measured time a little bit different (might not influence very much on results)

• in our case: we starts stopwatch in the beging Task WriteBatchAsync() method
• in your case: you measure only time needed for making request https://github.com/googleapis/google-cloud-dotnet/blob/master/apis/Google.Cloud.Firestore/Google.Cloud.Firestore.IntegrationBenchmarks/Program.cs#L124

1. Our pool was much more simpler:

EventHubClient GetEventHubClient()
        {
            var i = _rand.Next(0, _eventHubClients.Length);
            return _eventHubClients[i];
        }

1. Each firestoredb was created differently in our case;

FirestoreDb CreateFirestoreDb()
            {
                var client = new FirestoreClientBuilder { JsonCredentials = credentialsJson }.Build();

                return FirestoreDb.Create(projectName, client);
            }

Please remember that we observed differences when the pooling was turned on. Higher latency was observed when we used single firestoredb.