question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Select before insert

See original GitHub issue

Hello!

I’m trying to switch from 0.3.2-patch7 to 0.3.2-patch10 and at the same time replace the usage of ru.yandex.clickhouse with com.clickhouse.

I noticed before each INSERT you’re trying to perform prefetch column types, for example here - ClickHouseConnectionImpl.getTableColumns(). Yes, I investigated the code and understand the reason but is there ability to disable such behaviour? In that case you will use appropriate ClickHouseValue implementations based on the provided parameters or a custom implementation of the ClickHouseValue. (For example, at the moment there is no ability to customize SQL representation for inserting values.)

Why I’m asking about that? Firstly, because I know the types of data being inserted and don’t want extra interaction with the database.

Thanks

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:6

github_iconTop GitHub Comments

1reaction
zhicwucommented, Jul 29, 2022

How can I pass credentials (username and password)?

Yes, you can add credentials in ClickHouseNode using its builder function, which helps when you need to connect to multiple nodes using different credentials. You can also set default credentials at client level, for example:

ClickHouseClient client = ClickHouseClient.builder().defaultCredentials(...)...;
client.connect(<node>).query("select 1").executeAndWait();

Can I rely that the ClickHouseClient instance is thred-safe?

Yes it’s thread safe. However, you probably don’t have to create many client instances. Instead, you may reuse the client instance and create multiple ClickHouseRequest for queries. ClickHouseRequest is not thread-safe as of now.

1reaction
zhicwucommented, Jul 27, 2022

Yes, the input() function works well but to be honest the query syntax in this case is quite verbose.

I agree. I wish ClickHouse supports prepared statement so that we don’t have to do this.

It’ll be great if you mention about the input() function in the project readme/wiki.

There’s examples for batch insert at here but I guess it’s hard to notice. Moving them to wiki is probably better.

Are there other alternatives besides the input() function.

If the data volume is not huge and you don’t mind to keep them in memory(on ClickHouse server), you may try this but it will only make things unnecessarily complex.

As I understand, JDBC Driver is just a wrapper over Client API? And we can use it directly.

Yes, JDBC driver is a wrapper of Java client, and Java client hides implementation details like data format and protocol. Of course you can use Java client (examples) or more native approach to access ClickHouse like this. Alternatively, you can blend usage of JDBC and Java client, for instance: statement.unwrap(ClickHouseRequest.class).query("select 1").executeAndWait(), but it’s not recommended because the code is no longer portable.

What is recommended way to work with ClickHouse? Are there any conceptual benefits to use JDBC (in terms of performance and resource management)?

It depends. If you prefer stable API and may change to a different database in the future, you’d better stay with JDBC and avoid non-standard syntax in SQL queries. If performance is critical and you can tolerate API changes(before 0.4.0), it’s encouraged to use Java client. Performance-wise, JDBC is slower and it uses more memory compare to Java client. Below is the result took from here.

SELECT number FROM system.numbers_mt LIMIT 500000000

Name Time RAM
curl (C, HTTP, no deserialization) 3.7s 10M
clickhouse-client (Java, HTTP) 6.4s 121M
clickhouse-jdbc (Java, HTTP) 7.2s 120M
Read more comments on GitHub >

github_iconTop Results From Across the Web

Always check with SELECT before INSERT?
You could use WHERE NOT EXISTS to check new values before insert a new record. INSERT INTO <table> ( field1, field2, field3 )...
Read more >
Spring Boot / Hibernate Select's before insert - Stack Overflow
When I'm trying to insert a row in the following table, Hibernate does a SELECT before each insert and I'd like to avoid...
Read more >
INSERT INTO SELECT statement overview and examples
This article covers the INSERT INTO SELECT statement along with its syntax, examples and use cases.
Read more >
MySQL BEFORE INSERT Trigger Explained by Practical ...
Introduction to MySQL BEFORE INSERT triggers​​ In this syntax: First, specify the name of the trigger that you want to create in the...
Read more >
The (real) difference between the 'SELECT ... INTO' and ...
SELECT '. Because the 'INSERT … SELECT' inserts data into an existing table, it is slower and requires more resources due to the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found