Fn::Sub for SQS event source breaks

Using sqs event with Fn::Sub for the arn property breaks the package step. When I create my queue inside the resources section, and want to reference it, it breaks with the following error:

  TypeError: EventSourceArn.split is not a function
      at /...[snip].../serverless/lib/plugins/aws/package/compile/events/sqs/index.js:69:37

Which makes sense, because it breaks on the logic to try and get the queueName to add a queue to the resources itself (which I don’t want). I think the whole block should be in an if/else or try/catch maybe.

https://github.com/serverless/serverless/blob/8791cdacb75c84a2e08c5639abf769e915968288/lib/plugins/aws/package/compile/events/sqs.js#L60-L72

I currently fixed my issue by rewriting to use Fn::GetAtt but there is a very specific use case for Fn::Sub that can be very valuable. Also, I think it should just accept any valid Arn reference, as defined by the definitions (which also allows for Ref):

https://github.com/serverless/serverless/blob/8791cdacb75c84a2e08c5639abf769e915968288/lib/plugins/aws/provider.js#L261-L269

serverless.yml

service: myFooService

provider:
  name: aws
  runtime: python3.8
  stage: ${env:STAGE, 'beta'}
  region: eu-west-1
  timeout: 30

functions:
  fooFunction:
    handler: foo.bar
    reservedConcurrency: 10
    timeout: 30
    events:
      - sqs:
          arn:
            Fn::Sub: "#{FooQueue.Arn}"
          batchSize: 25 # max events per run
          maximumBatchingWindow: 10

resources:
  FooQueue:
    Type: AWS::SQS::Queue

serverless package output

  Type Error ---------------------------------------------

TypeError: EventSourceArn.split is not a function
     at /...[snip].../node_modules/serverless/lib/plugins/aws/package/compile/events/sqs/index.js:69:37
     at /...[snip].../node_modules/serverless/lib/plugins/aws/package/compile/events/sqs/index.js:70:15
     at Array.forEach (<anonymous>)
     at /...[snip].../node_modules/serverless/lib/plugins/aws/package/compile/events/sqs/index.js:44:28
     at Array.forEach (<anonymous>)
     at AwsCompileSQSEvents.compileSQSEvents (/...[snip].../node_modules/serverless/lib/plugins/aws/package/compile/events/sqs/index.js:34:47)
     at /...[snip].../node_modules/serverless/lib/classes/PluginManager.js:521:55
     at tryCatcher (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
     at Object.gotValue (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/reduce.js:168:18)
     at Object.gotAccum (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/reduce.js:155:25)
     at Object.tryCatcher (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/util.js:16:23)
     at Promise._settlePromiseFromHandler (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/promise.js:547:31)
     at Promise._settlePromise (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/promise.js:604:18)
     at Promise._settlePromise0 (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/promise.js:649:10)
     at Promise._settlePromises (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/promise.js:729:18)
     at _drainQueueStep (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/async.js:93:12)
     at _drainQueue (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/async.js:86:9)
     at Async._drainQueues (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/async.js:102:5)
     at Immediate.Async.drainQueues [as _onImmediate] (/...[snip].../node_modules/serverless/node_modules/bluebird/js/release/async.js:15:14)
     at processImmediate (internal/timers.js:456:21)
     at process.topLevelDomainCallback (domain.js:137:15)

Installed version

Framework Core: 2.15.0 (local)
Plugin: 4.3.0
SDK: 2.3.2
Components: 3.4.3

[pgrzesik]

Thank you for your response @svdgraaf. Thanks for clarification, that makes more sense now to me. I’m still wondering if the same couldn’t be possible to achieve via Join. The only tricky part is properly constructing the queueName for queueLogicalId when using Sub (or Join for that matter), but I think otherwise we should be able to support it. We’d be happy to accept a PR that adds support for Sub 👍

PS.

If the arn is a non-string (Import, Join, GetAtt, Ref, Sub), no queue resource should be created

Framework doesn’t create SQS queue automatically in any case, so that’s already covered

[pgrzesik]

Hello @svdgraaf 👋 Thank you for a very detailed report.

Also, I think it should just accept any valid Arn reference, as defined by the definitions (which also > allows for Ref)

I believe here we should update the schema definition to properly reflect what is actually possible when it comes to defining ARN for sqs event type. While Ref is currently allowed as per schema definition, I’m not aware of any proper way to obtain SQS ARN via Ref - for "AWS::SQS::Queue" resource type, Ref returns the URL of the Queue, not the ARN (reference: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sqs-queues.html). Do you know of any use case that would benefit from supporting Ref here?

I currently fixed my issue by rewriting to use Fn::GetAtt but there is a very specific use case for > Fn::Sub that can be very valuable.

What specific use case do you have in mind? While !Sub '${QueueResource.Arn} could potentially do the trick, I believe !GetAtt QueueResource.Arn feels a little bit more natural. The subtle problem with Sub is that it makes it a bit harder to derive queueName from it. Is there another use case for !Sub that I’ve missed?