Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

java.net.SocketTimeoutException: Read timed out when querying hive

See original GitHub issue

i’m using presto 0206 and i get read timeout when trying to query hive table after about 35 seconds, querying the same from hive cli works this only happens on tables with a lot of partitions

presto:default> select * from hive.default.passenger_demand limit 5;
Query 20191120_183710_00036_a5shx failed: 10.80.1.91: java.net.SocketTimeoutException: Read timed out
com.facebook.presto.spi.PrestoException: 10.80.1.91: java.net.SocketTimeoutException: Read timed out
	at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastore.getPartitionColumnStatistics(ThriftHiveMetastore.java:273)
	at com.facebook.presto.hive.metastore.thrift.BridgingHiveMetastore.getPartitionColumnStatistics(BridgingHiveMetastore.java:90)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.loadPartitionColumnStatistics(CachingHiveMetastore.java:341)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.access$100(CachingHiveMetastore.java:62)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore$2.loadAll(CachingHiveMetastore.java:156)
	at com.google.common.cache.CacheLoader$1.loadAll(CacheLoader.java:201)
	at com.google.common.cache.LocalCache.loadAll(LocalCache.java:4242)
	at com.google.common.cache.LocalCache.getAll(LocalCache.java:4205)
	at com.google.common.cache.LocalCache$LocalLoadingCache.getAll(LocalCache.java:5161)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.getAll(CachingHiveMetastore.java:224)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.getPartitionColumnStatistics(CachingHiveMetastore.java:314)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.loadPartitionColumnStatistics(CachingHiveMetastore.java:341)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.access$100(CachingHiveMetastore.java:62)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore$2.loadAll(CachingHiveMetastore.java:156)
	at com.google.common.cache.CacheLoader$1.loadAll(CacheLoader.java:201)
	at com.google.common.cache.LocalCache.loadAll(LocalCache.java:4242)
	at com.google.common.cache.LocalCache.getAll(LocalCache.java:4205)
	at com.google.common.cache.LocalCache$LocalLoadingCache.getAll(LocalCache.java:5161)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.getAll(CachingHiveMetastore.java:224)
	at com.facebook.presto.hive.metastore.CachingHiveMetastore.getPartitionColumnStatistics(CachingHiveMetastore.java:314)
	at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.getPartitionColumnStatistics(SemiTransactionalHiveMetastore.java:197)
	at com.facebook.presto.hive.statistics.MetastoreHiveStatisticsProvider.getPartitionsStatistics(MetastoreHiveStatisticsProvider.java:259)
	at com.facebook.presto.hive.statistics.MetastoreHiveStatisticsProvider.getPartitionsStatistics(MetastoreHiveStatisticsProvider.java:246)
	at com.facebook.presto.hive.statistics.MetastoreHiveStatisticsProvider.getTableStatistics(MetastoreHiveStatisticsProvider.java:76)
	at com.facebook.presto.hive.HiveMetadata.getTableStatistics(HiveMetadata.java:400)
	at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorMetadata.getTableStatistics(ClassLoaderSafeConnectorMetadata.java:176)
	at com.facebook.presto.metadata.MetadataManager.getTableStatistics(MetadataManager.java:354)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitTableScanWithPredicate(CoefficientBasedCostCalculator.java:189)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitTableScan(CoefficientBasedCostCalculator.java:182)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitTableScan(CoefficientBasedCostCalculator.java:84)
	at com.facebook.presto.sql.planner.plan.TableScanNode.accept(TableScanNode.java:135)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.lambda$visitSources$4(CoefficientBasedCostCalculator.java:270)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Collections$2.tryAdvance(Collections.java:4717)
	at java.util.Collections$2.forEachRemaining(Collections.java:4725)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSources(CoefficientBasedCostCalculator.java:271)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSource(CoefficientBasedCostCalculator.java:276)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitLimit(CoefficientBasedCostCalculator.java:248)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitLimit(CoefficientBasedCostCalculator.java:84)
	at com.facebook.presto.sql.planner.plan.LimitNode.accept(LimitNode.java:87)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.lambda$visitSources$4(CoefficientBasedCostCalculator.java:270)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Collections$2.tryAdvance(Collections.java:4717)
	at java.util.Collections$2.forEachRemaining(Collections.java:4725)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSources(CoefficientBasedCostCalculator.java:271)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitExchange(CoefficientBasedCostCalculator.java:161)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitExchange(CoefficientBasedCostCalculator.java:84)
	at com.facebook.presto.sql.planner.plan.ExchangeNode.accept(ExchangeNode.java:196)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.lambda$visitSources$4(CoefficientBasedCostCalculator.java:270)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Collections$2.tryAdvance(Collections.java:4717)
	at java.util.Collections$2.forEachRemaining(Collections.java:4725)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSources(CoefficientBasedCostCalculator.java:271)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitExchange(CoefficientBasedCostCalculator.java:161)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitExchange(CoefficientBasedCostCalculator.java:84)
	at com.facebook.presto.sql.planner.plan.ExchangeNode.accept(ExchangeNode.java:196)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.lambda$visitSources$4(CoefficientBasedCostCalculator.java:270)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Collections$2.tryAdvance(Collections.java:4717)
	at java.util.Collections$2.forEachRemaining(Collections.java:4725)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSources(CoefficientBasedCostCalculator.java:271)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSource(CoefficientBasedCostCalculator.java:276)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitLimit(CoefficientBasedCostCalculator.java:248)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitLimit(CoefficientBasedCostCalculator.java:84)
	at com.facebook.presto.sql.planner.plan.LimitNode.accept(LimitNode.java:87)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.lambda$visitSources$4(CoefficientBasedCostCalculator.java:270)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Collections$2.tryAdvance(Collections.java:4717)
	at java.util.Collections$2.forEachRemaining(Collections.java:4725)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSources(CoefficientBasedCostCalculator.java:271)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitSource(CoefficientBasedCostCalculator.java:276)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.copySourceCost(CoefficientBasedCostCalculator.java:262)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitOutput(CoefficientBasedCostCalculator.java:114)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator$Visitor.visitOutput(CoefficientBasedCostCalculator.java:84)
	at com.facebook.presto.sql.planner.plan.OutputNode.accept(OutputNode.java:82)
	at com.facebook.presto.cost.CoefficientBasedCostCalculator.calculateCostForPlan(CoefficientBasedCostCalculator.java:80)
	at com.facebook.presto.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:143)
	at com.facebook.presto.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:122)
	at com.facebook.presto.execution.SqlQueryExecution.doAnalyzeQuery(SqlQueryExecution.java:363)
	at com.facebook.presto.execution.SqlQueryExecution.analyzeQuery(SqlQueryExecution.java:342)
	at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:274)
	at com.facebook.presto.execution.QueuedExecution.lambda$start$1(QueuedExecution.java:62)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException: 10.80.1.91: java.net.SocketTimeoutException: Read timed out
	at com.facebook.presto.hive.metastore.thrift.Transport.rewriteException(Transport.java:91)
	at com.facebook.presto.hive.metastore.thrift.Transport.access$000(Transport.java:29)
	at com.facebook.presto.hive.metastore.thrift.Transport$TTransportWrapper.readAll(Transport.java:168)
	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:362)
	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:284)
	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:191)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partitions_statistics_req(ThriftHiveMetastore.java:3000)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions_statistics_req(ThriftHiveMetastore.java:2987)
	at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastoreClient.getPartitionColumnStatistics(ThriftHiveMetastoreClient.java:153)
	at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastore.lambda$getPartitionColumnStatistics$8(ThriftHiveMetastore.java:260)
	at com.facebook.presto.hive.metastore.thrift.HiveMetastoreApiStats.lambda$wrap$0(HiveMetastoreApiStats.java:42)
	at com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:138)
	at com.facebook.presto.hive.metastore.thrift.ThriftHiveMetastore.getPartitionColumnStatistics(ThriftHiveMetastore.java:258)
	... 109 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at com.facebook.presto.hive.metastore.thrift.Transport$TTransportWrapper.readAll(Transport.java:165)
	... 120 more

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

4reactions
arhimondrcommented, Jul 30, 2020

It is not in the doc unfortunately. Here is the property definition in the source code: https://github.com/prestodb/presto/blob/master/presto-hive-metastore/src/main/java/com/facebook/presto/hive/MetastoreClientConfig.java#L63

Feel free to open a PR updating the documentation.

On Wed, Jul 29, 2020 at 5:17 AM HanHui notifications@github.com wrote:

@arhimondr https://github.com/arhimondr i can’t find hive.metastore-timeout from document from there https://prestodb.io/docs/current/connector/hive.html. can you provide the doc, thx~

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/prestodb/presto/issues/13728#issuecomment-665545187, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKQDLFKUKQFLME7LY4ZDYDR57SMLANCNFSM4JPYHH7A .

0reactions
georgewfishercommented, Jan 27, 2022

I have had this exact situation and confirmed setting time timeout in the Hive catalog config solved this problem for me: hive.metastore-timeout=128s

Read more comments on GitHub >

github_iconTop Results From Across the Web

Frequent "Read timed out" error from Hive, using C... - 65858
The titled SocketTimeoutException occurs when the thrift-client in hiveConnection object is in the process of actively reading sql results from hive server2 ( ......
Read more >
Hive: Read timeout exception - Stack Overflow
This means the drop operation of the table was not able to get complete at Metadata level. (In general - DROP operation deletes...
Read more >
Read timed out" while running mapping involving Hive objects ...
The encountered issue occurs when the Hive server has large number of active connections at the time of mapping executions.
Read more >
hive.metastore-timeout doesn't take effect, socket read timeout ...
I have set hive.metastore-timeout = 3m, but still my query failed for ReadTimeOut in less than 3 ... PrestoException: hive-metastore-server:9083: java.net.
Read more >
1305397 - Presto queries are failing - Bugzilla@Mozilla
PrestoException: ip-172-31-27-197.us-west-2.compute.internal: java.net.SocketTimeoutException: Read timed out at com.facebook.presto.hive.metastore.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Hyphens in column names fail in where clause

Next page

Huge time for aggregation in postgresql