Unable to use basic filter with collection (was: Inconsistent behavior with SQL connector and message split)

This is a…

[ ] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[X] Bug report  
[ ] Documentation issue or request

Description

I have following integration:

1. SQL - Periodic invocation - Select * from contact
2. AMQ - Publish messages to some queue

In this scenario, the list from SQL connector is automatically split into multiple messages for some reason even when I didn’t specify the split step. So when I have 2 contacts in the database, I receive 2 messages in the queue.

Now I want to add a basic filter to the integration - since the split magic is being done, I might not need the explicit “split” step, so I create:

1. SQL - Periodic invocation - Select * from contact
2. Basic filter - last_name contains something (now I know something is wrong because I don't have the autocomplete for the field)
3. AMQ - Publish messages to some queue

Obviously, the integration fails with:

org.apache.camel.spring.boot.CamelSpringBootInitializationException: java.lang.IllegalArgumentException: java.lang.IllegalStateException: No step properties defined for rule filter step
	at org.apache.camel.spring.boot.RoutesCollector.onApplicationEvent(RoutesCollector.java:250) ~[camel-spring-boot-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.spring.boot.RoutesCollector.onApplicationEvent(RoutesCollector.java:57) ~[camel-spring-boot-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:883) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:144) ~[spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:546) ~[spring-context-4.3.19.RELEASE.jar!/:4.3.19.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) ~[spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) [spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) [spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) [spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) [spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) [spring-boot-1.5.16.RELEASE.jar!/:1.5.16.RELEASE]
	at io.syndesis.example.Application.main(Application.java:13) [classes!/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_191]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_191]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [project-0.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [project-0.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [project-0.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:595) [project-0.1-SNAPSHOT.jar:na]
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalStateException: No step properties defined for rule filter step
	at io.syndesis.integration.runtime.sb.IntegrationRuntimeAutoConfiguration$1.beforeApplicationStart(IntegrationRuntimeAutoConfiguration.java:94) ~[integration-runtime-springboot-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	at org.apache.camel.spring.boot.RoutesCollector.onApplicationEvent(RoutesCollector.java:152) ~[camel-spring-boot-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	... 24 common frames omitted
Caused by: java.lang.IllegalStateException: No step properties defined for rule filter step
	at io.syndesis.integration.runtime.handlers.RuleFilterStepHandler.getFilterExpression(RuleFilterStepHandler.java:51) ~[integration-runtime-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	at io.syndesis.integration.runtime.handlers.AbstractFilterStepHandler.handle(AbstractFilterStepHandler.java:36) ~[integration-runtime-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	at io.syndesis.integration.runtime.handlers.RuleFilterStepHandler.handle(RuleFilterStepHandler.java:30) ~[integration-runtime-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	at io.syndesis.integration.runtime.IntegrationRouteBuilder.configureFlow(IntegrationRouteBuilder.java:174) ~[integration-runtime-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	at io.syndesis.integration.runtime.IntegrationRouteBuilder.configure(IntegrationRouteBuilder.java:104) ~[integration-runtime-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	at org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:462) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:402) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:383) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.impl.DefaultCamelContext$1.call(DefaultCamelContext.java:1027) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.impl.DefaultCamelContext$1.call(DefaultCamelContext.java:1024) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:3270) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at org.apache.camel.impl.DefaultCamelContext.addRoutes(DefaultCamelContext.java:1024) ~[camel-core-2.21.0.fuse-730042.jar!/:2.21.0.fuse-730042]
	at io.syndesis.integration.runtime.sb.IntegrationRuntimeAutoConfiguration$1.beforeApplicationStart(IntegrationRuntimeAutoConfiguration.java:92) ~[integration-runtime-springboot-1.6-SNAPSHOT.jar!/:1.6-SNAPSHOT]
	... 25 common frames omitted

Probably the culprit is https://github.com/syndesisio/syndesis/issues/562 , but I think it does not make sense now since we have the dedicated split

[dongniwang]

In that case, can we add this issue to the Collection Enhancement Epic (if there’s one?) so we can look at this in 7.4. @christophd @heiko-braun

[KurtStam]

OK I will hide filter for a collection, and add a task to the ‘Collection Enhancement Epic’ for 7.4.