question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incorrect parsing of complex types (Arrays and Tuples)

See original GitHub issue

Here are some cases that I’ve found (rs is a ResultSet instance):

  1. Tuple:

    select ('1', 2) as a;
    

    can only be extracted as String:

    rs.getMetaData.getColumnTypeName(1) = "Tuple(String, UInt8)"
    rs.getObject(1) = "('1',2)"
    
  2. Nested arrays:

    select [[1,2], [3,4]] as a;
    

    cannot be extracted as Array:

    rs.getMetaData.getColumnTypeName(1) = "Array(Array(UInt8))"
    rs.getObject(1) // Method threw 'java.lang.RuntimeException' exception.
                               // Parse exception: ByteFragment{[[[1,2],[3,4]]], start=0, len=13}
    rs.getArray(1) // Method threw 'java.lang.IllegalStateException' exception.
    
  3. Tuples nested in Array:

    select [('a',1), ('b', 2)] as a;
    

    strange parsing behavior when parsed as Array:

    rs.getMetaData.getColumnTypeName(1) = Array(Tuple(String, UInt8))
    rs.getArray(1)={ru.yandex.clickhouse.ClickHouseArray}:
           array = {java.lang.String[4]@3644} 
           0 = "('a'"
           1 = "1)"
           2 = "('b'"
           3 = "2)"
    

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:2
  • Comments:9 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
serebrsergcommented, Jun 2, 2017

Thanks. Currently the driver has just a basic arrays support. All of cases you listed are expected. I’ll check what can be done on that.

1reaction
zhicwucommented, Oct 21, 2021

I cannot find ClickhouseTuple implementation in ClickHouse JDBC as of version 0.3.1-patch, the tuple may a suitable type for compound type.

Most data types including Tuple and mixed use of Array, Map and Tuple etc. are supported on develop branch. But it’s still experimental and only works with gRPC and RowBinary format at the moment, for example:

// create table test_table(
//   array_column Array(Tuple(UInt8, LowCardinality(String))),
//   array_tuple Tuple(Map(String, String), String)
// )engine=Memory

// insert into test_table values([(1,'a'), (2,'b')], ({'a':'1','b':'2'},'c'))

ClickHouseNode server = ClickHouseNode.of("localhost", ClickHouseProtocol.GRPC, 9100, "default");
try (ClickHouseClient client = ClickHouseClient.newInstance(server.getProtocol());
    ClickHouseResponse resp = client.connect(server).format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
      .query("select * from test_table").execute().get()) {
    for (ClickHouseRecord r : resp.records()) {
      List<?>[] array = (List<?>[]) r.getValue(0).asObject();
      List<?> tuple = r.getValue(1).asObject(List.class);

      Map<String, String> map = (Map<String, String>) tuple.get(0);
      ...
    }
}

JDBC driver will be enhanced accordingly.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python - mypy error regarding complex type List[Tuple[...]?
1 Answer 1 ... The problem is [tag, x, y] . Mypy does not recognize a type for "3-element list of string, int,...
Read more >
Querying arrays with complex types and nested structures
Your source data often contains arrays with complex data types and nested structures. Examples in this section show how to change element's data...
Read more >
Lists and Tuples in Python
The list is the first mutable data type you have encountered. Once a list has been created, elements can be added, deleted, shifted,...
Read more >
TypeScript 3.0: Exploring Tuples and the Unknown Type - Auth0
Let's delete the incorrect example from our code and move forward with the understanding that, with tuples, the order of values is critical....
Read more >
Macros with modifiers - master - Boost C++ Libraries
BOOST_VMD_RETURN_TYPE, return the type of data parsing any tuple-like ... list or an invalid array, whose parsing as the more specific type will...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found