Select before insert

Hello!

I’m trying to switch from 0.3.2-patch7 to 0.3.2-patch10 and at the same time replace the usage of ru.yandex.clickhouse with com.clickhouse.

I noticed before each INSERT you’re trying to perform prefetch column types, for example here - ClickHouseConnectionImpl.getTableColumns(). Yes, I investigated the code and understand the reason but is there ability to disable such behaviour? In that case you will use appropriate ClickHouseValue implementations based on the provided parameters or a custom implementation of the ClickHouseValue. (For example, at the moment there is no ability to customize SQL representation for inserting values.)

Why I’m asking about that? Firstly, because I know the types of data being inserted and don’t want extra interaction with the database.

Thanks

[zhicwu]

How can I pass credentials (username and password)?

Yes, you can add credentials in ClickHouseNode using its builder function, which helps when you need to connect to multiple nodes using different credentials. You can also set default credentials at client level, for example:

ClickHouseClient client = ClickHouseClient.builder().defaultCredentials(...)...;
client.connect(<node>).query("select 1").executeAndWait();

Can I rely that the ClickHouseClient instance is thred-safe?

Yes it’s thread safe. However, you probably don’t have to create many client instances. Instead, you may reuse the client instance and create multiple ClickHouseRequest for queries. ClickHouseRequest is not thread-safe as of now.

[zhicwu]

Yes, the input() function works well but to be honest the query syntax in this case is quite verbose.

I agree. I wish ClickHouse supports prepared statement so that we don’t have to do this.

It’ll be great if you mention about the input() function in the project readme/wiki.

There’s examples for batch insert at here but I guess it’s hard to notice. Moving them to wiki is probably better.

Are there other alternatives besides the input() function.

If the data volume is not huge and you don’t mind to keep them in memory(on ClickHouse server), you may try this but it will only make things unnecessarily complex.

As I understand, JDBC Driver is just a wrapper over Client API? And we can use it directly.

Yes, JDBC driver is a wrapper of Java client, and Java client hides implementation details like data format and protocol. Of course you can use Java client (examples) or more native approach to access ClickHouse like this. Alternatively, you can blend usage of JDBC and Java client, for instance: statement.unwrap(ClickHouseRequest.class).query("select 1").executeAndWait(), but it’s not recommended because the code is no longer portable.

What is recommended way to work with ClickHouse? Are there any conceptual benefits to use JDBC (in terms of performance and resource management)?

It depends. If you prefer stable API and may change to a different database in the future, you’d better stay with JDBC and avoid non-standard syntax in SQL queries. If performance is critical and you can tolerate API changes(before 0.4.0), it’s encouraged to use Java client. Performance-wise, JDBC is slower and it uses more memory compare to Java client. Below is the result took from here.

SELECT number FROM system.numbers_mt LIMIT 500000000

Name	Time	RAM
curl (C, HTTP, no deserialization)	3.7s	10M
clickhouse-client (Java, HTTP)	6.4s	121M
clickhouse-jdbc (Java, HTTP)	7.2s	120M