javacc parser is very slow to parse sql

I executed a batch insert SQL, but the execution was very slow. I took a look at Flame Graph and the sql parsing took most of the time. I looked at the source code of this project, which uses the parser generated by javacc. Will this have a significant impact on performance? I hope someone can help improve it.

my sql like this:

insert into trade_info (id, ....)
values 
(111, ...),
(222, ...),
(333, ...),
...
-- 1000 rows

[holmofy]

Sorry, I tested it again and found that using the native batch and using httpClient is similar.

For the previous slow of batch insert, the core reason is that the spring framework call connect.getMetaData() while some fields of the inserted data are null. And ClickHouseConnectionImpl.getMetaData() will use LogProxy reflection proxy to record logs.

I think the LogProxy can be optimized. There should be many frameworks that call ClickHouseConnectionImpl.getMetaData(). And reflection logging really affects performance.

[zhicwu]

Thanks again for the clarification. So to summarize:

1. SQL parser’s overhead is too high when dealing with large SQL
2. getMetaData should be optimized as it slows down JdbcTemplate for null-check - not sure why they use the method so frequent but looks like it can be cached
3. Batch insertion is inconvenient(compare to JdbcTemplate) and it can be optimized(~10% overhead)

Now batch insertion at this point is based on text-based format, so I expect the performance will be improved after switching to binary format. I’ll address the last two in 0.3.1 release, and the first in 0.4.0.