question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Will the performance of sharding-jdbc be optimized?

See original GitHub issue

Question

I have test a simple sql with version 4.0.0-RC1, found it’s too slow compared to version 2.0.3.

4.0.0-RC1 run result(10 times):

Statement select test(ms): 
  13  11  14  13  18  12   7  18  10  11
PreparedStatement select test(ms): 
   3   2   3   3   2   2   2   4   2   1

2.0.3 run result(10 times) :

Statement select test(ms): 
   2   2   3   2   3   6   2   2   3   3
PreparedStatement select test(ms): 
   2   2   5   4   2   2   2   2   1   3

Simple Test Demo

dependency

        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.0.9</version>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.36</version>
        </dependency>

        <dependency>
            <groupId>io.shardingjdbc</groupId>
            <artifactId>sharding-jdbc-core</artifactId>
            <version>2.0.3</version>
        </dependency>

        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-core</artifactId>
            <version>4.0.0-RC1</version>
        </dependency>

Prepare SQL

create database test_d_0;
create database test_d_1;
CREATE TABLE test_d_0.`ts_order_0000` (`id` BIGINT NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`));
CREATE TABLE test_d_0.`ts_order_0001` (`id` BIGINT NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`));
CREATE TABLE test_d_1.`ts_order_0002` (`id` BIGINT NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`));
CREATE TABLE test_d_1.`ts_order_0003` (`id` BIGINT NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`));

Java Code

public class ShardingPerformanceTest {

    public static void main(String[] args) throws Exception {
        String shardingConfig = loadFromResource("sharding-config-performance-simple-test.yaml");
        // for version 2.x.x
//        DataSource dataSource = ShardingDataSourceFactory.createDataSource(shardingConfig.getBytes("utf-8"));
        // for version 4.0.0
        DataSource dataSource = YamlShardingDataSourceFactory.createDataSource(shardingConfig.getBytes("utf-8"));

        // exclude first time
        executeWithStatement(dataSource, "select * from ts_order where id = 0");
        executeWithPrepareStatement(dataSource, "select * from ts_order where id = ?", 0);

        System.out.println("Statement select test(ms): ");
        for (int i = 0; i < 10; i++) {
            System.out.printf("%4d",
                    executeWithStatement(dataSource, "select * from ts_order where id = " + i));
        }
        System.out.println();

        System.out.println("PreparedStatement select test(ms): ");
        for (int i = 0; i < 10; i++) {
            System.out.printf("%4d",
                    executeWithPrepareStatement(dataSource, "select * from ts_order where id = ?", i));
        }
        System.out.println();
    }

    private static long executeWithStatement(DataSource dataSource, String sql) throws SQLException{
        Connection connection  = dataSource.getConnection();
        long begin = System.currentTimeMillis();
        Statement st = connection.createStatement();
        st.execute(sql);
        long executeTime = System.currentTimeMillis() - begin;
        st.close();
        connection.close();
        return executeTime;
    }

    private static long executeWithPrepareStatement(DataSource dataSource, String sql, Object... args) throws SQLException{
        Connection connection  = dataSource.getConnection();
        long begin = System.currentTimeMillis();
        PreparedStatement ps = connection.prepareStatement(sql);
        for (int i = 0; i < args.length; i++) {
            ps.setObject(i + 1, args[i]);
        }
        ps.execute();
        long executeTime = System.currentTimeMillis() - begin;
        ps.close();
        connection.close();
        return executeTime;
    }

    private static String loadFromResource(String path) throws Exception{
        BufferedReader reader = new BufferedReader(new InputStreamReader(Class.class
                .getResourceAsStream("/" + path), "utf-8"));
        StringBuffer sb = new StringBuffer();
        CharBuffer charBuffer = CharBuffer.allocate(1024);
        for (int count = reader.read(charBuffer); count > 0; count = reader.read(charBuffer)) {
            sb.append(charBuffer.flip());
        }
        return sb.toString();
    }

}

Sharding config

sharding-config-performance-simple-test.yaml

dataSources:
  ds_0: !!com.alibaba.druid.pool.DruidDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/test_d_0
    username: root
    password: root135
  ds_1: !!com.alibaba.druid.pool.DruidDataSource
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://localhost:3306/test_d_1
    username: root
    password: root135

shardingRule:
  tables:
    ts_order:
      actualDataNodes: ds_0.ts_order_000${0..1},ds_1.ts_order_000${2..3}
      databaseStrategy:
        inline:
          shardingColumn: id
          algorithmExpression: ds_${new BigDecimal(id).abs().divideAndRemainder(4)[1].longValue().intdiv(2)}
      tableStrategy:
        inline:
          shardingColumn: id
          algorithmExpression: ts_order_${String.format("%04d",new BigDecimal(id).abs().divideAndRemainder(4)[1].longValue())}

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
terrymanucommented, May 13, 2019

If user use statement, we can assume they don’t care about the performance

0reactions
lwtdevcommented, May 14, 2019

Thanks very much for your reply.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Will the performance of sharding-jdbc be optimized? #2365
I have test a simple sql with version 4.0.0-RC1, found it's too slow compared to version 2.0.3. 4.0.0-RC1 run result(10 times):.
Read more >
Executor Engine: Performance Optimization Showcase with ...
The number of shards in the test tables is relatively large but the overall performance is still improved by about 4 times. Theoretically,...
Read more >
Performance Test - Apache ShardingSphere
The performance of Sharding-JDBC,Sharding-Proxy and MySQL would be compared here. INSERT & UPDATE & DELETE which regarded as a set of associated operation ......
Read more >
ShardingSphere-Proxy Performance with PostgreSQL is ...
This article introduces some of the performance optimizations at the code level, and showcases the optimized results of ShardingSphere-Proxy ...
Read more >
26 JDBC Support for Database Sharding - Oracle Help Center
The sharding data source supports only the JDBC Thin driver. · The sharding data source does not support some Oracle JDBC extension APIs...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found