Roadmap 2021

0.2.x

Focus on bug fixes, small enhancements, and backward compatibility...

• 0.2.5
  • switch to github actions for consistency
  • use testcontainer for integration test
• 0.2.6
  • enable retry for idempotent queries(as workaround of host failed to respond)
  • new sql parser
  • use basic auth instead of query parameters for authentication
• 0.2.7 - TBD(in case any critical issue)

0.3.x

Focus on new features, code clean up, and abstraction which may break existing interfaces/APIs…

Previous releases...

• 0.3.0
  • BREAKING CHANGE: drop JDK7 support
  • BREAKING CHANGE: remove Guava dependency (UnsignedLong is removed, please use long(faster) or BigInteger(slower) instead for UInt64) Note: shaded jar is now ~3.65MB(was 7.19MB in 0.2.6, and 5.68MB in 0.2.4).
  • JDBC 4.2 support
  • more data types (including aliases) like IPv4, IPv6, DateTime64, *Int128, *Int256, Decimal256 and Map Note: UInt128 will be supported soon on server side.
  • RoaringBitmap support - please use latest RoaringBitmap
  • restructure code (clickhouse-jdbc for JDBC compliance, and clickhouse-*client for efficiency and consistent behaviors like any other clickhouse client, see #570)
  • performance test (clickhouse-jdbc vs. clickhouse4j vs. clickhouse-native-jdbc vs. mariadb-java-client)
  • CI enhancement: checkstyle, spellcheck & SonarCloud
• 0.3.1
  • BREAKING CHANGE: remove deprecated stuff Note: will also drop fallback of SQL parsing
  • BREAKING CHANGE: exclude roaringbitmap in uber jar and remove jitpack.io maven repository - see #603
  • multi-statement support - only return the last result

Ongoing releases...

• 0.3.2
  • JPMS support along with multi-release jars

     19M	target/clickhouse-jdbc-0.3.2-SNAPSHOT-all.jar
     18M	target/clickhouse-jdbc-0.3.2-SNAPSHOT-grpc.jar
    664K	target/clickhouse-jdbc-0.3.2-SNAPSHOT-http.jar
    960K	target/clickhouse-jdbc-0.3.2-SNAPSHOT-javadoc.jar
    2.7M	target/clickhouse-jdbc-0.3.2-SNAPSHOT-shaded.jar
    428K	target/clickhouse-jdbc-0.3.2-SNAPSHOT.jar
    

  • introduce abstract module clickhouse-client, experimental clickhouse-grpc-client, and HttpURLConnection-based clickhouse-http-client
  • named parameter support(only available in clickhouse-client)
  • support RowBinary* format and more data types(Geo types, Date32, Tuple, Nested, mixed use of Array/Tuple/Map etc.)
  • new JDBC driver(com.clickhouse.jdbc.ClickHouseDriver) built on top of clickhouse-client Note: both old and new drivers will co-exist in 0.3.x series and the old one will be removed starting from 0.4.
  • show schema of remote datasources(when JDBC bridge is available)
  • fix timezone and DateTime64 related issues
  • adaptive integration test against local testcontainer or a remote server, and categorize cases under different groups
  • replace jackson-databind and jackson-core by gson
  • enhance benchmarks to cover most JDBC drivers and data types
  • alternative implementation for http(s) protocol(JDK 11 HttpClient)

[zhicwu]

These are great inputs! IMO, put the repository name aside, clickhouse-jdbc is both JDBC driver and Java client for ClickHouse 😃 Implementing JDBC interfaces simplifies integration, while as a Java client, it should stick with the clickhouse-way - simple to use and efficient in runtime. Alternatively, we can simply treat it as a JDBC wrapper over Java client for ClickHouse, meaning we can choose suitable interface to talk to ClickHouse using JVM languages, depending on what we need.

I’m with you on JDBC compliance. That’s why I created #545 and suggested to introduce a new setting. On the other hand, we probably don’t have to completely follow JDBC standards - not only because ClickHouse has its own limitation(like transaction support, server-side cursor etc.), but also to save time so that we can focus on things that matter the most(like more data types support, streaming, caching, stability etc.). It seems we’d better restructure the code early using multiple modules/packages.

I think we can start to create more issues/feature requests and add them into the release plan, starting from 0.3.0. I’m sure we can fill the gap between 0.4 and 1.0 quickly 😉 And of course it’s a plan that we can change as needed.

Some more comments:

• JDBC compliance
  1. Agree that error handling and empty implementation should be improved.
  2. Is eclipse’s TCK based on the one created by Sun? If it works on JDK8, we should definitely add a new github workflow for that.
  3. Yes we can review data type mappings as well as concepts like catalog/schema, and get them documented.
  4. I saw 4.2 but what about 4.3? Also I’m thinking to break the large PR into several smaller PRs, so that it’s easier to review and we can deliver them gradually in multiple releases.
• Cleaner code
  1. We have github actions for CI/CD but they’re far from good. Yes, we do need to apply checkstyle template, coverage analysis and benchmarking.
  2. RE: integration test, since we’re using TestNG, I guess we just need to categorize them under different groups. And of course a base class is helpful, so that we don’t have to add ClickHouseContainerForTest.beforeSuite() in setUp method 😉
• Control non-JDBC API
  1. Yes, we need to be more careful on modifiers. We should avoid case like #499.
  2. I didn’t see issue related to OSGi/Java9 but we can use package/maven module to separate interface and implementation.
  3. Besides a unified client API for hiding protocol details, we may still need to expose API for specific client like http/grpc/native for better performance/usability.
• Performance
  1. ClickHouse feeds response in chunks, which in a way is a forward-only “cursor” that we can use.
  2. Benchmark should be added to measure hot areas, so that it’s clear if a PR will improve performance or not.
  3. Thought about voting but now I’d say requirements came from github issues and Telegram/Slack 😃 Definitely good idea to setup a meeting to go over the list and exchange thoughts before 0.3.0. Give me a few days to digest all these information and let me see what I can do to kick-off future development.

[enqueue]

Hi @zhicwu , I am excited to see a renewed activity in this project. I think it is important that the official JDBC driver is maintained. Please let me know if there is anything I can help with.