Using AWS CDK in pytest unit tests receives jsii.errors.JSIIError: Maximum call stack size exceeded

AWS Support case ID: 6378067991

🐛 Bug Report

What is the problem?

I am working on a pytest test suite for a Python application that creates complex networks using AWS CDK. Most of the tests involve creating a small CDK app, adding some objects and matching the generated CloudFormation code using app.synth() with mock CloudFormation code, defined upfront.

Currently, some of our tests are failing when the full test suite is run. The tests do not fail when executed in isolation, or when only the module is executed, only when the full test suite is executed.

When using the Python debugger and pytest, I’ve located where the error occurs. It is in the following code: https://github.com/aws/jsii/blob/master/packages/jsii-python-runtime/src/jsii/_kernel/__init__.py#L71 in jsii in lines 71 to lines 83. The program has entered this loop due to a call to node.apply_aspect(Tag(key, value)) which should apply tags to the Construct being constructed.

Reproduction Steps

Currently, I don’t have any easy reproduction steps without sharing confidential code.

Verbose Log

jsii.errors.JavaScriptError:
  RangeError: Maximum call stack size exceeded
      at KernelHost.processRequest (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6230:18)
      at KernelHost.run (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6137:14)
      at /venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6137:45
      at KernelHost.processRequest (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6233:16)
      at KernelHost.run (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6137:14)
      at /venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6137:45
      at KernelHost.processRequest (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6233:16)
      at KernelHost.run (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6137:14)
      at /venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6137:45
      at KernelHost.processRequest (/venv/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6233:16)

self = <jsii._kernel.providers.process._NodeProcess object at 0x10d913710>
request = GetRequest(objref=CreateResponse(ref='@aws-cdk/core.Construct@11247'), property='node'), response_type = <class 'jsii._kernel.types.GetResponse'>

    def send(
        self, request: KernelRequest, response_type: Type[KernelResponse]
    ) -> KernelResponse:
        req_dict = self._serializer.unstructure(request)
        data = json.dumps(req_dict, default=jdefault).encode("utf8")

        # Send our data, ensure that it is framed with a trailing \n
        self._process.stdin.write(b"%b\n" % (data,))
        self._process.stdin.flush()

        resp: _ProcessResponse = self._serializer.structure(
            self._next_message(), _ProcessResponse_R
        )

        if isinstance(resp, _OkayResponse):
            return self._serializer.structure(resp.ok, response_type)
        elif isinstance(resp, _CallbackResponse):
            return resp.callback
        else:
>           raise JSIIError(resp.error) from JavaScriptError(resp.stack)
E           jsii.errors.JSIIError: Maximum call stack size exceeded

../../../venv/lib/python3.7/site-packages/jsii/_kernel/providers/process.py:316: JSIIError

Environment

• CDK CLI Version: 1.2.0 (the issue also occurs in the latest build 1.5.0)
• Module Version: jsii 0.14.2`
• OS: OSX Mojave
• Language: Python

Other information

I have a feeling it has something to do with the JSII kernel being overloaded due to all the calls that occur when testing. Most tests will create a new App, Stack and multiple other Constructs when executing.

[roystchiang]

I think when you import the library, it creates a global variable, and initializes it with some pseudo-tokens. As you create stacks, more token are added to this variable, and it is not cleaned up when a new App is created.

[SomayaB]

@bverhoeve That’s great! I will close the issue then. Please let us know if you experience any other issues.