Not able to deserialize ClickHouseBitmap for too many bitmaps(>4294967295)!
See original GitHub issueclickhouse-server version:22.2.2.1 clickhouse-jdbc version:0.3.2-patch8
an error occurs when I fetch the bitmap of the next. even if I adjust the order of index_val, the second data will still report an error.
jdbc sql: select index_name, index_val, length(bitmapToArray(cst_bitmap)) as bitmap_count, cst_bitmap fromindex_bitmap_mergetree orderbyindex_name,index_val jdbc sql: select index_name, index_val, length(bitmapToArray(cst_bitmap)) as bitmap_count, cst_bitmap fromindex_bitmap_mergetree orderbyindex_name,index_val desc
test code
public static void main(String[] args) throws SQLException {
String url = "jdbc:clickhouse://192.168.70.131:8123";
Properties properties = new Properties();
ClickHouseDataSource dataSource = new ClickHouseDataSource(url, properties);
// insert(dataSource, 2000);
searchBitmap(dataSource);
}
public static void searchBitmap(ClickHouseDataSource dataSource) {
try (Connection connection = dataSource.getConnection("default", "");
Statement statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery("" +
"select " +
" index_name," +
" index_val," +
" length(bitmapToArray(cst_bitmap)) as bitmap_count, " +
" cst_bitmap " +
"from index_bitmap_mergetree " +
"order by index_name, index_val desc")) {
while (resultSet.next()) {
String indexName = resultSet.getString(1);
String indexVal = resultSet.getString(2);
int bitmapCount = resultSet.getInt(3);
Roaring64NavigableMap roaringBitmap = (Roaring64NavigableMap) resultSet.getObject(4, ClickHouseBitmap.class).unwrap();
System.out.printf("indexName:%s, indexVal:%s, size:%d%n", indexName, indexVal, bitmapCount);
roaringBitmap.stream().forEach(i -> System.out.print(i + ","));
System.out.println();
}
} catch (SQLException e) {
e.printStackTrace();
}
}
error
Exception in thread "main" java.lang.IllegalStateException: Not able to deserialize ClickHouseBitmap for too many bitmaps(>4294967295)!
at com.clickhouse.client.data.ClickHouseBitmap.deserialize(ClickHouseBitmap.java:372)
at com.clickhouse.client.data.ClickHouseBitmap.deserialize(ClickHouseBitmap.java:339)
at com.clickhouse.client.data.BinaryStreamUtils.readBitmap(BinaryStreamUtils.java:210)
at com.clickhouse.client.data.ClickHouseRowBinaryProcessor$MappedFunctions.lambda$buildMappingsForAggregateFunctions$0(ClickHouseRowBinaryProcessor.java:178)
at com.clickhouse.client.data.ClickHouseRowBinaryProcessor$MappedFunctions.lambda$buildMappingsForAggregateFunctions$2(ClickHouseRowBinaryProcessor.java:185)
at com.clickhouse.client.data.ClickHouseRowBinaryProcessor$MappedFunctions.deserialize(ClickHouseRowBinaryProcessor.java:464)
at com.clickhouse.client.data.ClickHouseRowBinaryProcessor.readAndFill(ClickHouseRowBinaryProcessor.java:496)
at com.clickhouse.client.ClickHouseDataProcessor$1.next(ClickHouseDataProcessor.java:122)
at com.clickhouse.client.ClickHouseDataProcessor$1.next(ClickHouseDataProcessor.java:108)
at com.clickhouse.jdbc.ClickHouseResultSet.next(ClickHouseResultSet.java:708)
at com.yatop.clickhouse.JdbcDemo.searchBitmap(JdbcDemo.java:39)
at com.yatop.clickhouse.JdbcDemo.main(JdbcDemo.java:25)
Issue Analytics
- State:
- Created a year ago
- Comments:10 (3 by maintainers)
Top Results From Across the Web
AggregateFunction(groupBitmap,UInt64) support #698 - GitHub
No able to deserialize ClickhouseBitmap for too many bitmaps ... it more than 4,294,967,295? And which version of ClickHouse are you using?
Read more >Bitmap Functions | ClickHouse Docs
Bitmap functions work for two bitmaps Object value calculation, it is to return new bitmap or cardinality while using formula calculation, such as...
Read more >ClickHouse Documentation
If a query uses a primary key and does not select too many rows to process (hundreds of thousands), and does not use...
Read more >Not able to deserialize ClickHouseBitmap for too ... - bytemeta
IllegalStateException: Not able to deserialize ClickHouseBitmap for too many bitmaps(>4294967295)! at com.clickhouse.client.data.
Read more >How can i perform AND operation on multiple bitmaps
1 Answer 1 · I found that groupbitmapand function can be implemented clickhouse.com/docs/en/sql-reference/aggregate-functions/… – lant. Apr 7 at ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@zhicwu I have tested it in my local environment. The bug is caused by the bitmap reading 3 bytes too many, resulting in the following field reading misalignment
Thanks for confirming @wzqiang1332! It will be part of 0.3.2-patch11 which hopefully can be released by Friday, before I start to play Xenoblade 3 😃