Multiple Imports Fail To Synth

Description of the bug:

When working on a project with multiple imports (such as the example for CRDs), the cdk8s import function works smoothly to generate the full set of definitions, but cdk8s synth (or invoking app.synth() from code) results in a jsii.errors.JavaScriptError due to conflicting definitions of the generated package in nodejs. I’ve observed this both in the crd sample and in an internal project, so the behavior seems to be consistent for any project using multiple imports.

There seem to also be some secondary bugs in the crd example. The paths generated via cdk8s imports did not match the coded import paths in main.py. I needed to change them from

from imports import jenkins
from imports import clusterinstallation

to

from imports.jenkins.io import jenkins
from imports.mattermost.com import clusterinstallation

Reproduction Steps:

1. cd examples/python/crd
2. cdk8s import
3. Make the above changes to main.py
4. cdk8s synth

Error Log:

jsii.errors.JavaScriptError: 
  Error: Type 'generated.ClusterInstallation' not found

Full Error

jsii.errors.JavaScriptError: 
  Error: Type 'generated.ClusterInstallation' not found
      at Kernel._typeInfoForFqn (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:8200:19)
      at Kernel._findCtor (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7898:31)
      at Kernel._create (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7920:33)
      at Kernel.create (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7666:21)
      at KernelHost.processRequest (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7446:28)
      at KernelHost.run (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7384:14)
      at Immediate._onImmediate (/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7387:37)
      at processImmediate (internal/timers.js:439:21)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./main.py", line 49, in <module>
    MyChart(app, "crd-python")
  File "/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_runtime.py", line 66, in __call__
    inst = super().__call__(*args, **kwargs)
  File "./main.py", line 44, in __init__
    mattermost_license_secret="mattermost-license"
  File "/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_runtime.py", line 66, in __call__
    inst = super().__call__(*args, **kwargs)
  File "/Users/ghart/Projects/github/cdk8s/examples/python/crd/imports/mattermost/com/clusterinstallation/__init__.py", line 33, in __init__
    jsii.create(ClusterInstallation, self, [scope, name, options])
  File "/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_kernel/__init__.py", line 229, in create
    interfaces=[iface.__jsii_type__ for iface in getattr(klass, "__jsii_ifaces__", [])],
  File "/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_kernel/providers/process.py", line 333, in create
    return self._process.send(request, CreateResponse)
  File "/Users/ghart/.local/share/virtualenvs/crd-3rAX9go0/lib/python3.7/site-packages/jsii/_kernel/providers/process.py", line 318, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: Type 'generated.ClusterInstallation' not found
Error: command "pipenv run ./main.py " at /Users/ghart/Projects/github/cdk8s/examples/python/crd returned a non-zero exit code 1
    at ChildProcess.<anonymous> (/usr/local/Cellar/cdk8s/0.25.0/libexec/lib/node_modules/cdk8s-cli/lib/util.js:24:27)
    at Object.onceWrapper (events.js:300:26)
    at ChildProcess.emit (events.js:210:5)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)

Environment:

For my internal project, the build environment is configured via docker. Here is a Dockerfile that will reproduce the issue with the crd example:

FROM ubuntu:bionic

RUN apt update -y
RUN apt install -y curl git python3-pip python3-venv zlib1g-dev libssl-dev \
    && curl -sL https://deb.nodesource.com/setup_12.x | bash - \
    && apt install -y nodejs \
    && npm install -g cdk8s-cli \
    && pip3 install pipenv \
    && curl https://pyenv.run | bash

ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV PATH=$PATH:/root/.pyenv/bin

RUN mkdir /src \
    && cd /src \
    && git clone https://github.com/awslabs/cdk8s.git \
    && cd /src/cdk8s/examples/python/crd \
    && pyenv install 3.7.8 \
    && pipenv install \
    && pipenv install constructs cdk8s \
    && cdk8s import \
    && sed -i 's,from imports import clusterinstallation,from imports.mattermost.com import clusterinstallation,' main.py \
    && sed -i 's,from imports import jenkins,from imports.jenkins.io import jenkins,' main.py

# This is where the error occurs
RUN cd /src/cdk8s/examples/python/crd \
    && cdk8s synth

• Framework Version:

  ?> cdk8s --version
  0.26.0
  
  ?> nodejs --version
  v12.18.3
  
  ?> pipenv run python --version
  Python 3.7.8
  

• OS:
  • The above uses ubuntu:bionic I’ve also reproduced this running natively on MacOS and in a docker file based on rhel.

Other:

I’ve been trying to learn the interplay between cdk8s, jsii, and (just discovered) jsii-srcmak. From what I can tell, the issue is arising because the name field in the package.json inside of the generated nodejs package is generated for all packages generated with jsii-srcmak here. My best guess so far is that the generated node package is conflicting and whichever gets imported first in python is the one that gets correctly registered with the node process, and all others that are imported from python after that are not correctly imported.

---

This is 🐛 Bug Report

[campionfellin]

Indeed, it does look like this line: jsii-srcmak/src/compile.ts#L51 is the issue here.

Replaced it with a random string, and re-built everything and it seemed to work.

I’ll look for a way to replace that generated with something that makes sense, and I’ll update the Python examples. Thanks for the detailed bug report @gabe-l-hart !

[gabe-l-hart]

Thank you for the investigation! I’ll keep an eye out here for an updated release with the new changes.