Need some recommendations when Gmail API batch request returns 429 responses

hello,

I have a multitenant application which does email synchronization with local database to cache user emails. Service is using GmailService for email synchronization with my local database. When cache is invalidated I need to do full synchronization.

For this first I call (in the loop using NextPageToken for 2nd and further pages):

service.Users.Messages.List("me")

to find out message id’s and then for each received message I build a batch request to get all details:

_request.Queue<Message>(
    _service.Users.Messages.Get("me", messageId),
    (message, error, index, response) =>
    {
        if (error != null)
        {
            var id = _ids[index];
            _messages.Add(new GmailMessage { Id = id, Error = error });
        }
        else
        {
            _messages.Add(new GmailMessage { Id = message.Id, Message = message });
        }
    });

Sometimes for some of these messages in the batch I get errors like:

Failed to receive message with id [xxxxxxxxxxxxxxx]. Error: Google.Apis.Requests.RequestError
Too many concurrent requests for user [429]
Errors [
	Message[Too many concurrent requests for user] Location[ - ] Reason[rateLimitExceeded] Domain[global]
]

Since only some messages are failing (like 60 out of 100 in the batch). I am not sure how to deal with these?

• How long should I wait before retrying?
• Should I do retry only for failed messages or for a whole batch. Not sure about this since I am storing HistoryId for later to find out changes since last sync, and in this case not clear if I can use it because of these errors. Lets imagine I will store it - but I won’t succeed in retrying for some reason - it will mean that I have HistoryId which assumes my failed messages are included, but actually it is not. So probably I cannot continue with the sync until I get retries solved?

Still its a bit strange to get those half failing because of this error as I am sending a “single” request, still I understand that each of these are individual ones at the end…

Thanks for any recommendations 😃

[amanda-tarafa]

As an FYI: A note on the recommended batch size has been added on the general batch documentation: https://developers.google.com/gmail/api/guides/batch#overview.

Thanks for flagging.

[amanda-tarafa]

I’ve created an internal issue for the note on the recommended batch size to be included on the general batch documentation. Thanks for flagging.