Each vault export write (ExportWritesPerMinutePerProject) quota is counted 10x

Google Vault API issue tracker has dismissed the issue, 2, 3, and most recently pointed toward python client as the issue (other users experience it Java, haven’t test JS yet) and asked to open issue here.

Description

Every export appears to be counted 10 times toward the “Export Writes per minute” quota

This is easily visualized on the GCP Project’s Quota page. Reproducible when exporting with a SINGLE export.

Requirements

• Python3
• Google SDK libraries installed via pip (Follow dev Python QuickStart guide)
• Sample email account with data (empty/new accounts may not reproduce error)

What steps will reproduce the problem?

1. Create new GCP Project
2. Use API tab and search library for “vault”
3. Enable - https://console.cloud.google.com/apis/library/vault.googleapis.com for project
4. Create OAuth Desktop client
5. Download json key and paste local path into script (line 11)
6. Run script: e.g. python3 samplescript.py [e-mail]
7. Observe error in exports (20 export limit reached after 2 exports)

Expected

When exporting any fewer than 20 exports the RATE_LIMIT_EXCEEDED error should not be encountered.

SAMPLE CODE

Most of this is from Google’s sample Python Quickstart docs and example export documentation

from __future__ import print_function
import os.path, sys, time
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials

SCOPES = ['https://www.googleapis.com/auth/ediscovery']

# Path to local OAuth 2 client id file (create with "Desktop" type)
oauth_2_client_id_file = '/Users/grantgriffith/Downloads/client_secret_########.json'

def create_matter(service, email_address):
  matter_content = {
      'name': '{} - {} - DEV_TEST'.format(str(email_address),time.asctime())
  }
  matter = service.matters().create(body=matter_content).execute()
  return matter

def create_mail_account_held_data_export(service, export_name, matter_id, email_to_search):
    mail_query = {
      'corpus': 'MAIL',
      'dataScope': 'ALL_DATA',
      'searchMethod': 'ACCOUNT',
      'accountInfo': {
          'emails': [email_to_search]
      },
      "startTime": "2021-04-01T14:04:41Z", # Optional, limits export size
      "endTime": "2021-05-01T14:04:41Z", # Optional, limits export size
    }
    mail_export_options = {
        'exportFormat': 'MBOX',
        'showConfidentialModeContent': True
        }
    wanted_export = {
        'name': export_name + " MAIL Export",
        'query': mail_query,
        'exportOptions': {
            'mailOptions': mail_export_options
            }
        }
    return service.matters().exports().create(
        matterId=matter_id, body=wanted_export).execute()
  
def main():
    creds = None
    # The file token.json stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('token.json'):
        creds = Credentials.from_authorized_user_file('token.json', SCOPES)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                oauth_2_client_id_file, SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.json', 'w') as token:
            token.write(creds.to_json())

    vault_service = build('vault', 'v1', credentials=creds)

    #Create new test Google Vault Matter
    new_matter = create_matter(vault_service, sys.argv[1])
    print("{} - CREATED, Status - {}".format(new_matter['name'], new_matter['state']))

    id = new_matter['matterId']
    email = sys.argv[1]

    #Create 1st export
    mail_export = create_mail_account_held_data_export(vault_service, "FIRST", id, email)
    print(mail_export['name'], mail_export['status'])

    #Create 2nd export
    mail_export2 = create_mail_account_held_data_export(vault_service, "SECOND", id, email)
    print(mail_export2['name'], mail_export2['status'])

    #Create 3rd export
    mail_export3 = create_mail_account_held_data_export(vault_service, "THIRD", id, email)
    print(mail_export3['name'], mail_export3['status'])

    #Create 4th export
    mail_export4 = create_mail_account_held_data_export(vault_service, "FOURTH", id, email)
    print(mail_export4['name'], mail_export4['status'])

if __name__ == '__main__':
    main()

HTTP Error response from my new test project that reproduces the error:

googleapiclient.errors.HttpError: 
<HttpError 429 when requesting 
https://vault.googleapis.com/v1/matters/e67ac8b7-1e55-49b9-9d9f-ddefa8233fb7/exports?alt=json returned 
"Quota exceeded for quota metric 'Export Writes' and limit 'Export Writes per minute' of service 'vault.googleapis.com' for consumer 'project_number:721746368665'.". 
Details: 
"[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "reason": "RATE_LIMIT_EXCEEDED",
    "domain": "googleapis.com",
    "metadata": {
      "service": "vault.googleapis.com",
      "consumer": "projects/721746368665",
      "quota_limit": "ExportWritesPerMinutePerProject",
      "quota_metric": "vault.googleapis.com/export_writes"
    }
  }
]">

[parthea]

I’m going to close this issue as “Working as Intended”, but please feel free to re-open this issue if you have additional questions.

If you have specific questions about quotas, you can reach out to the API team directly via this support page.

[parthea]

Hi @mngrant, I’ve heard back from the API team. This is working as intended. Each call to export.create costs as 10 towards your quota limit, whereas export.get costs as 1 towards your quota limit. Please visit this page to view your current quota. You can also request a quota increase. Note that this quota is a per minute quota, and is different from the limit of 20 concurrent exports.