question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to congiure --numPartitions and --partitionStrategy

See original GitHub issue

I want to know how to properly set numPartitions and partitionStrategy, I am using HBase, my data format is shapefile, a total of 7434km2

geowave config addindex -t spatial  -np 1024 -ps HASH fwm3-spatial
geowave config addstore  -t hbase -z server7:2181,server8:2181,server9:2181 --gwNamespace geowave.hwang fwm3
geowave ingest localtogw ./fwm.shp fwm3 fwm3-spatial
geowave gs addlayer -ws gepwave fwm3

I ause mapbox-vector slices, and Distributed environment also. But I found that the slicing speed is very slow, I guess this is related to numPartitions and partitionStrategy, so I would like to know how to set these parameters. As shown below. 1 I also configured Accumulo( pseudo-cluster), and GeoMesa , then imported the same data, I feel better.As shown below. 2

Is this related to Accumulo and Hbase?

Geoserver: 3

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:14 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
zzhnbcommented, Sep 14, 2018

I am sorry that I meet the same problem… why it’s so slowly when loading pbf(vectortile)with Hbase

0reactions
rfechercommented, Sep 6, 2019

there is an open distributed rendering issue which I think is where this issue left off

Read more comments on GitHub >

github_iconTop Results From Across the Web

Kafka Partitions: 3 Easy Steps to Create and Use - Hevo Data
Furthermore, you can decide the number of partitions while creating topics in Kafka Partition by executing commands in the command prompt.
Read more >
Spark Partitioning & Partition Understanding
Spark/PySpark partitioning is a way to split the data into multiple partitions so that you can execute transformations on multiple ...
Read more >
On Spark Performance and partitioning strategies - Medium
Number of partitions is calculated on the base on various parameters: spark.default.parallelism (value depends on used cluster manager, see ...
Read more >
Best practices and strategies for Kafka topic partitioning
In this post, we explain how the partitioning strategy for your producers depends on what your consumers will do with the data.
Read more >
How to Choose the Number of Topics/Partitions in a Kafka ...
A rough formula for picking the number of partitions is based on throughput. You measure the throughout that you can achieve on a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found