[Sync] Allow for partial push

First of all thank you for WatermelonDB, we’ve been playing around with using it instead of redux-offline and its a world of difference.

One of our use cases that I have a bit of trouble wrapping my head around is pushing changes to the backend. Our app is for making sales orders (you receive products from the backend, and turn them into orders in the client and send these to the backend). This creates 2 issues when it comes to pushing client changes to backend:

1. Drafts / Pushing partially

Usually you work on a sales order for a while before you are done. And you might want to double check the inventory or similar before pushing so you do a pull before approval. This means you might have orders that should not be pushed at all (because they are not ready). I guess you could fix this by adding an approved boolean field to your table and then filter orders that you send to the backend using orders.filter(order => order.approved) but that would not work since the sync mechanism assumes all records are pushed on a sync, and will blindly set status=synced on successful push. Which leads me to the second point

2. Allow for push response / server side validation

A very important point for us is that we have server side validation. E.g. an invoice has incremental invoice number but it might come from several clients so there is not way the client can add this. This will be added in the backend and is completely immutable. Another thing is that inventory levels are checked on the server. This means that you might send 10 orders to the backend where 2 will fail due to low inventory levels. But that does not mean the 8 others should fail.

So my thought is that why not add a response structure for a push? Something like

Request

{
    "orders": {
        "created": [
            { "id": "order-1", "customer_id": "4321", "error": null },
            { "id": "order-2", "customer_id": "4321", "error": null },
            { "id": "order-3", "customer_id": "4321", "error": null }
        ],
        "updated": [

        ],
        "deleted": [

        ]
    },
    "orders_lines": {
        "created": [
            { "id": "order-line-1", "order_id": "order-1", "product_id": "4321", "quantity": 1, "error": null },
            { "id": "order-line-2", "order_id": "order-2", "product_id": "4321", "quantity": 1, "error": null  },
            { "id": "order-line-3", "order_id": "order-3", "product_id": "4321", "quantity": 1, "error": null  }
        ],
        "updated": [

        ],
        "deleted": [

        ]
    }
}

Response

{
    "orders": {
        "success": [
            { "id": "order-1", "order_number": "4540", "created_at": "2018-01-01 12:00:00" },
            { "id": "order-3", "order_number": "4541", "created_at": "2018-01-01 12:00:00" }
        ],
        "errors": [
            { "id": "order-2", "error": "Not enough in stock" },
        ]
    },
    "orders_lines": {
        "success": [
            { "id": "order-line-1", "created_at": "2018-01-01 12:00:00" },
            { "id": "order-line-3", "created_at": "2018-01-01 12:00:00" }
        ],
        "errors": [
            { "id": "order-line-2", "error": "Not enough in stock"  }
        ]
    }
}

The idea here is that you allow the client to only mark successful ones as synced. All the objects in an error array will not be marked as synced. This would also allow for use-case (1) since I can ignore orders with approved=false and not include them in the success array. This means they would still be marked as unsynced in the database.

This also allows for partial updates with e.g. timestamps, error messages, invoice numbers or similar. Note that this behaviour could be opt-in by simply marking all as synced when returning empty status 200. I am by no means an expert in synchronization so I have no idea the problems this introduces, but I’d love to hear your thoughts around something like this?

[esbenp]

Yes, we are open for contributing. We are actually on faires this weekend trying out a new version of our app that uses the Just Sync Everything model, so we will definitely have some feedback next week. We definitely have some ideas already so lets talk shop soon 😃

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.