Max retry attempts and max wait time for hive metastore client should be configurable

I find the when ThriftHiveMetastore trying to connect to hive metastore, it always using the default DetryDriver whose max retry attempts is DEFAULT_RETRY_ATTEMPTS whose value is 10 and max retry time is DEFAULT_MAX_RETRY_TIME whose value is 30s; All that user could setup currently is the timeout by hive.metastore-timeout, but for example, when I want to set the hive.metastore-timeout to 3 minutes and want it to retry for 10 attempts when timeout happened, that is , waiting for 3 * 10 = 30 minutes when hive metastore didn’t respond or unstable, it is impossible because it is restricted by DEFAULT_MAX_RETRY_TIME which cannot be configured; I think it is necessary to make it configurable in the catalog configuration files;

@dain @findepi what’s your opinion ? If you think it is necessary , I will do that.

[electrum]

Note that RetryDriver has maxAttempts so you’ll need to do maxRetries + 1.

[electrum]

Sorry for the delayed response. I agree we should make this configurable. Let’s add a ThriftHiveMetastoreConfig class with these config properties:

• metastore.thrift.client.max-retries
• metastore.thrift.client.min-backoff-delay
• metastore.thrift.client.max-backoff-delay
• metastore.thrift.client.backoff-scale-factor
• metastore.thrift.client.max-retry-time

These config names are taken from DriftClientConfig in the Drift library (it would be bound with a prefix like metastore). If we ever switch the metastore client to Drift, the config names won’t have to change. You can see how we use this for the Thrift Connector.