sharding scaling didn't sync the increment data

Bug Report

Please answer these questions before submitting your issue. Thanks!

Which version of ShardingSphere did you use?

4.1.0 or 5.0.0 SNAPSHOT , MySQL 5.7.28

Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?

sharding scaling

Expected behavior

Sync increment data

Actual behavior

Does’n work

Reason analyze (If you can)

Maybe the handleUpdateRowsEvent doesn’t work well

Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.

before start this issue , I already ensure the binlog sync mode is ROW , and test table has PK

1. insert some test data into table source_schema.test
2. start sharding scaling, and post the scaling/job/start request
3. scaling start to sync the inventory data , this works well
4. during scaling sync the inventory data , try to update one of the data in source_schema.test , such as update test set amount = amount +1 where id = 123
5. wait for a moment , all inventory data have been transtered to new sharded database through proxy, but the update change doesn’t work

I grant the root user all privileges , and config root user in proxy.I’m not sure if there is any other configuration for database or proxy I should set up ?

I tried insert data ,during the inventory sync , if insert some new data , these could be sync to sharded part , but update data does’t work

[Lucas-307]

@taojintianxia hi, I find a easy way to do this.

f_user_id -> Long.parseLong(f_user_id)

shardingRule:
  tables:
    test:
      actualDataNodes: ds_$->{0..1}.test
      databaseStrategy:
        inline:
          shardingColumn: f_user_id
          algorithmExpression: ds_${Long.parseLong(f_user_id) % 2} # transfer to long
      logicTable: test

[taojintianxia]

@taojintianxia Hi, Sun Nianjun, there are 2 problems in this issue. First is config, logicTable must equals tablename. (actually, tablename means logicTable, you can remove directly.)

shardingRule:
  tables:
    test:
      actualDataNodes: ds_$->{0..1}.test
      databaseStrategy:
        inline:
          shardingColumn: f_user_id
          algorithmExpression: ds_${f_user_id % 2}
      logicTable: test # logicTable must equals tablename , or you can remove it.

Second is about mod() error. it’s because f_user_id is transfered to string, string can’t execute mod in sharding algorithm ${f_user_id % 2}.

groovy.lang.MissingMethodException: No signature of method: java.lang.String.mod() is applicable for argument types: (java.lang.Integer) values: [2]
Possible solutions: drop(int), find(), any(), find(groovy.lang.Closure), any(groovy.lang.Closure), use([Ljava.lang.Object;)

As you know, bigint transfered to string, is because we use ps.setObject to assemb data in jdbc statement, and mysql transfered it to string.

see: com.mysql.jdbc.PreparedStatement#setObject(int, java.lang.Object)

    public void setObject(int parameterIndex, Object parameterObj) throws SQLException {
        synchronized(this.checkClosed().getConnectionMutex()) {
            if (parameterObj == null) {
                this.setNull(parameterIndex, 1111);
            } else if (parameterObj instanceof Byte) {
                this.setInt(parameterIndex, ((Byte)parameterObj).intValue());
            } else if (parameterObj instanceof String) {
                this.setString(parameterIndex, (String)parameterObj);
            } else if (parameterObj instanceof BigDecimal) {
                this.setBigDecimal(parameterIndex, (BigDecimal)parameterObj);
            } else if (parameterObj instanceof Short) {
                this.setShort(parameterIndex, (Short)parameterObj);
            } else if (parameterObj instanceof Integer) {
                this.setInt(parameterIndex, (Integer)parameterObj);
            } else if (parameterObj instanceof Long) {
                this.setLong(parameterIndex, (Long)parameterObj);
            } else if (parameterObj instanceof Float) {
                this.setFloat(parameterIndex, (Float)parameterObj);
            } else if (parameterObj instanceof Double) {
                this.setDouble(parameterIndex, (Double)parameterObj);
            } else if (parameterObj instanceof byte[]) {
                this.setBytes(parameterIndex, (byte[])((byte[])parameterObj));
            } else if (parameterObj instanceof java.sql.Date) {
                this.setDate(parameterIndex, (java.sql.Date)parameterObj);
            } else if (parameterObj instanceof Time) {
                this.setTime(parameterIndex, (Time)parameterObj);
            } else if (parameterObj instanceof Timestamp) {
                this.setTimestamp(parameterIndex, (Timestamp)parameterObj);
            } else if (parameterObj instanceof Boolean) {
                this.setBoolean(parameterIndex, (Boolean)parameterObj);
            } else if (parameterObj instanceof InputStream) {
                this.setBinaryStream(parameterIndex, (InputStream)parameterObj, -1);
            } else if (parameterObj instanceof Blob) {
                this.setBlob(parameterIndex, (Blob)parameterObj);
            } else if (parameterObj instanceof Clob) {
                this.setClob(parameterIndex, (Clob)parameterObj);
            } else if (this.connection.getTreatUtilDateAsTimestamp() && parameterObj instanceof Date) {
                this.setTimestamp(parameterIndex, new Timestamp(((Date)parameterObj).getTime()));
            } else if (parameterObj instanceof BigInteger) {
// BigInteger treated as String, that's why bingint transfered to string in sql.
                this.setString(parameterIndex, parameterObj.toString());
            } else {
                this.setSerializableObject(parameterIndex, parameterObj);
            }
        }
    }

But In mysql, we can treated a string number as bigint like the sql, but not in ShardingSphere-Proxy.

insert into t_test(f_id, f_user_id,f_money) values(100,'1',1);

So, we decide to enhance proxy with string number, not in scaling only.

If you still interesting in it, you can try.

quite clearly, I undertand .