Not able to deserialize ClickHouseBitmap for too many bitmaps(>4294967295)!

clickhouse-server version:22.2.2.1 clickhouse-jdbc version:0.3.2-patch8

an error occurs when I fetch the bitmap of the next. even if I adjust the order of index_val, the second data will still report an error.

jdbc sql: select index_name, index_val, length(bitmapToArray(cst_bitmap)) as bitmap_count, cst_bitmap fromindex_bitmap_mergetree orderbyindex_name,index_val jdbc sql: select index_name, index_val, length(bitmapToArray(cst_bitmap)) as bitmap_count, cst_bitmap fromindex_bitmap_mergetree orderbyindex_name,index_val desc

test code

  public static void main(String[] args) throws SQLException {
      String url = "jdbc:clickhouse://192.168.70.131:8123";
      Properties properties = new Properties();
      ClickHouseDataSource dataSource = new ClickHouseDataSource(url, properties);
//        insert(dataSource, 2000);
      searchBitmap(dataSource);
  }

  public static void searchBitmap(ClickHouseDataSource dataSource) {
      try (Connection connection = dataSource.getConnection("default", "");
           Statement statement = connection.createStatement();
           ResultSet resultSet = statement.executeQuery("" +
                   "select " +
                   "  index_name," +
                   "  index_val," +
                   "  length(bitmapToArray(cst_bitmap)) as bitmap_count, " +
                   "  cst_bitmap " +
                   "from index_bitmap_mergetree " +
                   "order by index_name, index_val desc")) {
          while (resultSet.next()) {
              String indexName = resultSet.getString(1);
              String indexVal = resultSet.getString(2);
              int bitmapCount = resultSet.getInt(3);
              Roaring64NavigableMap roaringBitmap = (Roaring64NavigableMap) resultSet.getObject(4, ClickHouseBitmap.class).unwrap();
              System.out.printf("indexName:%s, indexVal:%s, size:%d%n", indexName, indexVal, bitmapCount);
              roaringBitmap.stream().forEach(i -> System.out.print(i + ","));
              System.out.println();
          }
      } catch (SQLException e) {
          e.printStackTrace();
      }
  }

error

Exception in thread "main" java.lang.IllegalStateException: Not able to deserialize ClickHouseBitmap for too many bitmaps(>4294967295)!
	at com.clickhouse.client.data.ClickHouseBitmap.deserialize(ClickHouseBitmap.java:372)
	at com.clickhouse.client.data.ClickHouseBitmap.deserialize(ClickHouseBitmap.java:339)
	at com.clickhouse.client.data.BinaryStreamUtils.readBitmap(BinaryStreamUtils.java:210)
	at com.clickhouse.client.data.ClickHouseRowBinaryProcessor$MappedFunctions.lambda$buildMappingsForAggregateFunctions$0(ClickHouseRowBinaryProcessor.java:178)
	at com.clickhouse.client.data.ClickHouseRowBinaryProcessor$MappedFunctions.lambda$buildMappingsForAggregateFunctions$2(ClickHouseRowBinaryProcessor.java:185)
	at com.clickhouse.client.data.ClickHouseRowBinaryProcessor$MappedFunctions.deserialize(ClickHouseRowBinaryProcessor.java:464)
	at com.clickhouse.client.data.ClickHouseRowBinaryProcessor.readAndFill(ClickHouseRowBinaryProcessor.java:496)
	at com.clickhouse.client.ClickHouseDataProcessor$1.next(ClickHouseDataProcessor.java:122)
	at com.clickhouse.client.ClickHouseDataProcessor$1.next(ClickHouseDataProcessor.java:108)
	at com.clickhouse.jdbc.ClickHouseResultSet.next(ClickHouseResultSet.java:708)
	at com.yatop.clickhouse.JdbcDemo.searchBitmap(JdbcDemo.java:39)
	at com.yatop.clickhouse.JdbcDemo.main(JdbcDemo.java:25)

[wzqiang1332]

@zhicwu I have tested it in my local environment. The bug is caused by the bitmap reading 3 bytes too many, resulting in the following field reading misalignment

[zhicwu]

Thanks for confirming @wzqiang1332! It will be part of 0.3.2-patch11 which hopefully can be released by Friday, before I start to play Xenoblade 3 😃