Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Offload data from bk to aws-s3 error on 2.5.2

See original GitHub issue

Describe the bug



the broker error as follows:

NarClassLoader@3a8dc651] info register aspect
13:36:59.190 [pulsar-ordered-OrderedExecutor-6-0] INFO  org.apache.bookkeeper.mledger.offload.OffloaderUtils - Found and loaded 1 offloaders
13:36:59.196 [pulsar-ordered-OrderedExecutor-6-0] ERROR - create ledgerOffloader failed for namespace public/functions Either s3ManagedLedgerOffloadRegion or s3ManagedLedgerOffloadServiceEndpoint must be set if s3 offload enabled
	at ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
	at$getManagedLedgerOffloader$4( ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
	at java.util.concurrent.ConcurrentHashMap.compute( ~[?:1.8.0_201]
	at ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
	at$getManagedLedgerConfig$33( ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
	at org.apache.bookkeeper.mledger.util.SafeRun$2.safeRun( [org.apache.pulsar-managed-ledger-2.5.2.jar:2.5.2]
	at [org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker( [?:1.8.0_201]
	at java.util.concurrent.ThreadPoolExecutor$ [?:1.8.0_201]
	at [io.netty-netty-common-4.1.48.Final.jar:4.1.48.Final]
	at [?:1.8.0_201]
Caused by: Either s3ManagedLedgerOffloadRegion or s3ManagedLedgerOffloadServiceEndpoint must be set if s3 offload enabled
	at ~[org.apache.pulsar-pulsar-broker-2.5.2.jar:2.5.2]
	... 10 more

To Reproduce Steps to reproduce the behavior:

  1. connfig conf/standalone.conf
  2. start pulsar 2.5.2 standalone

Expected behavior The broker can start normally

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

lukestephenson-zendeskcommented, Jun 11, 2020

I’ve also encountered the same issue with pulsar 2.5.2.

From reviewing commits, it looks like it was introduced here

Note that the ManagedLedgerOffloadDriver is copied across, but not other fields on the configuration.

That code has since been removed in, so fingers crossed this is fixed in pulsar 2.6.0.

jiazhaicommented, Jun 19, 2020

since 2.6.0 is released, would like to close this issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

AWS S3 offloader - StreamNative Hub
This tiered storage solution extends the storage capabilities of Pulsar by offloading data from Apache BookKeeper to scalable cloud-native storage or files ...
Read more >
Resolve errors uploading data to or downloading data from ...
I want to download data from Amazon Aurora and upload it to Amazon S3. How can I resolve an error I received while...
Read more >
Use AWS S3 offloader with Pulsar
Namespace policy can be configured to offload data automatically once a threshold is reached. The threshold is based on the size of data...
Read more >
WP Offload Media Changelog - Delicious Brains
Read all about the latest updates to WP Offload Media in our changelog. ... Improvement: Some common AWS S3 errors are caught and...
Read more >
S3 Error: The difference between the request time and the ...
I have error The difference between the request time and the current time is too large when call method amazons3.ListObjects
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found