Getting only first row from SQL connector

This is a…

[ ] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report  
[ ] Documentation issue or request

The problem

Seems that we’re only passing the first row of an result set down the exchange.

https://github.com/syndesisio/syndesis/blob/be740466d017a5d2c20ca84044f1583b734ea6a2/app/connector/sql/src/main/java/io/syndesis/connector/sql/customizer/SqlConnectorCustomizer.java#L71-L72

It might be because we wanted to split and process (map?) each row individually but I can’t find how this would be done in the code.

This is quite an issue for API provider-based integrations and API’s that would fetch more than one row from the database, say to implement GET /task endpoint.

To address this we should probably set the output data shape of the SQL connector to something like:

{
  "results": [
    {"col1":"data1"},
    {"col2":"data2"}
  ]
}

So this would mean change the dynamic data shape generation (meta) and change in the SqlConnectorCustomizer::doAfterProducer.

Expected behavior

Put all rows on the exchange.

Tasks involved / Steps to Reproduce

1. create timer to log integration (logging body)
2. add invoke SQL action, using sample db connection with select * from todo SQL statement
3. insert two or more rows into the sampledb.todo table
4. notice that only the first row is logged

[lburgazzoli]

I think we need to work on https://github.com/syndesisio/syndesis/issues/4337 to properly solve this. The idea is that the DataShape describe the shape of the elements of the collection and then we do add some metadata/tag to the DataShape definition to tell UI/DataMapper that the action produces a collection.

How this flag is consumed is not yet defined

[christophd]

So I put this to a test and to my amazement both versions

1. SQL connector just returning the first result (what we have today)
2. SQL connector returning the complete list of results

worked in the exact same way!

What I did:

• Create an integration with the current implementation
  • timer -> sql -> data-mapping -> google-sheets
  • As expected the SQL connector just returned the first row and this was mapped and sent to Google Sheets
• Update backend so SQL connector returns the complete result set as collection
• Update Syndesis with new implementation
• Restart the integration
  • The SQL connector returns the collection
  • Data mapper just maps the first entry and this is sent to Google Sheets
• Exact same behavior as before!

Also published the integration once again and it also worked as before.