Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Retry request for shard warning at Bootstrap using Akka Management

See original GitHub issue

Lagom Version (1.2.x / 1.3.x / etc)

1.4.1

API (Scala / Java / Neither / Both)

Scala

Operating System (Ubuntu 15.10 / MacOS 10.10 / Windows 10)

Alpine

JDK (Oracle 1.8.0_112, OpenJDK 1.8.x, Azul Zing)

java8

Library Dependencies

“org.postgresql” % “postgresql” % “42.2.1”

Expected Behavior

Request for shard of entities should wait for complete cluster boostrap.

Actual Behavior

Using akka management (with 1 ou 2 nodes in akka cluster). The Akka cluster is started and everything is working fine for newly created entities but previously created entities raise a Retry request for shard warning.

I used the default akka.cluster.sharding configuration. Do I need to change a timeout ?

[info] a.e.s.Slf4jLogger - Slf4jLogger started
[info] a.r.Remoting - Starting remoting
[info] a.r.Remoting - Remoting started; listening on addresses :[akka.tcp://connectors_bot_builder_app@10.62.23.205:24260]
[info] a.r.Remoting - Remoting now listens on addresses: [akka.tcp://connectors_bot_builder_app@10.62.23.205:24260]
[info] a.m.c.b.ClusterBootstrap - Bootstrap using default `akka.discovery` mechanism: DnsSrvServiceDiscovery
[info] a.i.AsyncDnsResolver - Using the following DNS nameservers: consul.service.consul/10.62.153.168:53
[info] a.m.AkkaManagement - Including HTTP management routes for ClusterHttpManagement
[info] a.m.AkkaManagement - Including HTTP management routes for ClusterBootstrap
[info] a.m.c.b.ClusterBootstrap - Got self contact point address: http://10.62.23.205:26817
[info] a.m.AkkaManagement - Binding Akka Management (HTTP) endpoint to: 0.0.0.0:19999
[info] a.m.c.b.ClusterBootstrap - Initiating bootstrap procedure using akka.discovery method...
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Locating service members; Lookup: tpocreau-connectors-bot-builder-app-akka-management
[info] a.m.AkkaManagement - Bound Akka Management (HTTP) endpoint to: 0.0.0.0:19999
[info] application - Creating Pool for datasource 'default'
[info] c.z.h.HikariDataSource - HikariPool-1 - Starting...
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: List()
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [0] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] c.z.h.HikariDataSource - HikariPool-1 - Start completed.
[info] p.a.d.HikariCPConnectionPool - datasource [default] bound to JNDI as DefaultDS
[info] p.a.d.DefaultDBApi - Database [default] connected at jdbc:postgresql://main-conversations-postgresql.service.consul:5432/conversations_bot_connector
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: List()
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [0] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] c.i.c.b.i.ConnectorsBotBuilderAppLoader$$anon$2 - BotEditorEventConsumer singleton started.
[warn] a.a.ActorSystemImpl - CircuitBreakers is deprecated, use CircuitBreakersPanel instead. This warning is probably caused by your service locator. If you are using a 3rd party service locator, upgrade your dependencies, otherwise this service locator could become incompatible with Lagom in future versions.
[info] play.api.Play - Application started (Prod)
[info] p.c.s.AkkaHttpServer - Listening for HTTP on /0.0.0.0:9000
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: List()
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [0] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: List()
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [0] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: List()
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [0] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Discovered [1] observation, which is less than the required [2], retrying (interval: 1.000 s)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [20] buffered messages.
[info] a.d.d.DnsSrvServiceDiscovery - Resolved Dns.Resolved: Vector(ServiceAddress(tcp,app-10-62-23-205.node.dev-eu_de_1.consul,10.62.23.205,26817), ServiceAddress(tcp,app-10-62-25-244.node.dev-eu_de_1.consul,10.62.25.244,23254))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Initiating contact-point probing, sufficient contact points: ResolvedTarget(10.62.23.205,Some(26817)), ResolvedTarget(10.62.25.244,Some(23254))
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Ensuring probing actor: contactPointProbe-10.62.23.205-26817
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Ensuring probing actor: contactPointProbe-10.62.25.244-23254
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 172.17.0.1:57702: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 10.62.25.244:42478: Contact Point returning 0 seed-nodes ([Set()])
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [40] buffered messages.
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 172.17.0.1:57702: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 10.62.25.244:42478: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 172.17.0.1:57702: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 10.62.25.244:42478: Contact Point returning 0 seed-nodes ([Set()])
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [40] buffered messages.
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 172.17.0.1:57702: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 10.62.25.244:42478: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 172.17.0.1:57702: Contact Point returning 0 seed-nodes ([Set()])
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [None], but no acknowledgement. Total [40] buffered messages.
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 10.62.25.244:42478: Contact Point returning 0 seed-nodes ([Set()])
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Contact point [http://10.62.25.244:23254] exceeded stable margin with no seed-nodes in sight. Considering whether this node is allowed to JOIN itself to initiate a new cluster.
[info] a.m.c.b.d.HeadlessServiceDnsBootstrap - Initiating new cluster, self-joining [akka.tcp://connectors_bot_builder_app@10.62.23.205:24260], as this node has the LOWEST address out of: [List(ResolvedTarget(10.62.23.205,Some(26817)), ResolvedTarget(10.62.25.244,Some(23254)))]! Other nodes are expected to locate this cluster via continued contact-point probing.
[info] c.l.l.i.p.c.ClusterStartupTaskActor - Executing cluster start task jdbcCreateTables.
[info] c.l.l.i.p.c.ClusterStartupTaskActor - Executing cluster start task slickOffsetStorePrepare.
[info] c.i.c.b.i.BotEditorEventsConsumer - Starting Stream
[info] c.l.l.i.p.c.ClusterStartupTaskActor - Cluster start task slickOffsetStorePrepare done.
[info] c.l.l.i.p.c.ClusterStartupTaskActor - Cluster start task jdbcCreateTables done.
[info] a.m.c.b.c.HttpClusterBootstrapRoutes - Bootstrap request from 10.62.25.244:42478: Contact Point returning 1 seed-nodes ([Set(ClusterMember(akka.tcp://connectors_bot_builder_app@10.62.23.205:24260,2144373385,Up,Set(dc-default)))])
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [Some(ActorSelection[Anchor(akka://connectors_bot_builder_app/), Path(/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator)])], but no acknowledgement. Total [40] buffered messages.
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [Some(ActorSelection[Anchor(akka://connectors_bot_builder_app/), Path(/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator)])], but no acknowledgement. Total [40] buffered messages.
[info] c.i.c.b.i.BotEditorEventsConsumer - Retrieved 11 scenarios with ids: List(dev-tpocreau-1_9_7f16deac-7603-4ac6-86f7-b571232e1f81, dev-tpocreau-1_13_edbd3212-daec-42f4-a78f-d3d99209bf08, dev-tpocreau-1_1_bfa6e470-5ae7-44d2-85df-de927e888d1a, dev-tpocreau-1_2_50459ad1-a2e1-423d-9f91-32a85e5e9efa, dev-tpocreau-1_3_2c78b1b0-259c-4e4a-a4d5-580adc2ab43d, dev-tpocreau-1_4_40440009-9035-4878-924a-bda84d7044d0, dev-tpocreau-1_5_a5836416-265a-4257-9be8-e8eed7ed6531, dev-tpocreau-1_6_ce79f4e9-d275-44ba-8d66-5d3c36871c81, dev-tpocreau-1_7_07ff1a22-f81c-4717-9c14-d58dadbfe661, dev-tpocreau-1_8_52b7e713-d4d5-47eb-a089-4ae949a397c6, dev-tpocreau-1_16_1ed48f6c-fb35-4172-8f5e-487fb76b47c4)
[warn] a.c.s.ShardRegion - Trying to register to coordinator at [Some(ActorSelection[Anchor(akka://connectors_bot_builder_app/), Path(/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator)])], but no acknowledgement. Total [40] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot12] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot11] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot14] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot13] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot10] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot1] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot0] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot3] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot2] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot5] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot19] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot4] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot7] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot6] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot9] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot16] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot15] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot8] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot18] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[warn] a.c.s.ShardRegion - Retry request for shard [connectors-bot-builder-app-bot17] homes from coordinator at [Actor[akka://connectors_bot_builder_app/system/sharding/kafkaProducer-dev-tpocreau-connectors-bot-builder-app-eventsCoordinator/singleton/coordinator#-383811339]]. [2] buffered messages.
[error] a.a.OneForOneStrategy - Ask timed out on [Actor[akka://connectors_bot_builder_app/user/slickOffsetStorePrepare-singletonProxy#-1531252869]] after [20000 ms]. Sender[null] sent message of type "com.lightbend.lagom.internal.persistence.cluster.ClusterStartupTaskActor$Execute$".
akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://connectors_bot_builder_app/user/slickOffsetStorePrepare-singletonProxy#-1531252869]] after [20000 ms]. Sender[null] sent message of type "com.lightbend.lagom.internal.persistence.cluster.ClusterStartupTaskActor$Execute$".
	at akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:595)
	at akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:605)
	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:866)
	at scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)
	at scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:864)
	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
[error] a.a.OneForOneStrategy - Ask timed out on [Actor[akka://connectors_bot_builder_app/user/slickOffsetStorePrepare-singletonProxy#-1531252869]] after [20000 ms]. Sender[null] sent message of type "com.lightbend.lagom.internal.persistence.cluster.ClusterStartupTaskActor$Execute$".
akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://connectors_bot_builder_app/user/slickOffsetStorePrepare-singletonProxy#-1531252869]] after [20000 ms]. Sender[null] sent message of type "com.lightbend.lagom.internal.persistence.cluster.ClusterStartupTaskActor$Execute$".
	at akka.pattern.PromiseActorRef$.$anonfun$defaultOnTimeout$1(AskSupport.scala:595)
	at akka.pattern.PromiseActorRef$.$anonfun$apply$1(AskSupport.scala:605)
	at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:140)
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:866)
	at scala.concurrent.BatchingExecutor.execute(BatchingExecutor.scala:109)
	at scala.concurrent.BatchingExecutor.execute$(BatchingExecutor.scala:103)
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:864)
	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
	at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)

Reproducible Test Case

Please provide a PR with a failing test.

If the issue is more complex or requires configuration, please provide a link to a project on Github that reproduces the issue.

Issue Analytics

  • State:open
  • Created 6 years ago
  • Comments:12 (8 by maintainers)

github_iconTop GitHub Comments

2reactions
pocmancommented, Mar 19, 2018

@ignasi35 I created a new issue here as requested. Thanks for you feedback.

0reactions
jropercommented, Feb 18, 2019

Here’s my 15 minute fix to this: #1743.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Default configuration - Documentation - Akka
Default configuration. Each Akka module has a reference.conf file with the default values. Make your edits/overrides in your application.conf .
Read more >
Configuration - Documentation - Akka
Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala.
Read more >
Cluster Sharding - Documentation - Akka
Shard a clustered compute process across the network with locationally transparent message routing using Akka Cluster Sharding.
Read more >
Cluster Usage • Akka Documentation
This document describes how to use Akka Cluster and the Cluster APIs. ... is available using the Akka Management project's module, Cluster Bootstrap....
Read more >
Cluster Sharding - Documentation - Akka
Warning. Don't use Cluster Sharding together with Automatic Downing, since it ... Too many shards will result in less efficient management of the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Connection to node -1 could not be established. Broker may not be available.

Next page

Recommend enabling protobuf serializer for Address/RemoteAddress