[Suggestion] A hook in aws-eks for cdk8s Chart

Is your feature request related to a problem? Please describe.

We would like to abstract the details of provisioning AWS infrastructure (EKS, etc.) from our customers (data scientists). They provide a Spark docker container with the application code and we run it for them. We are leveraging a new capability in Apache Spark so that we can run the job on EKS. We have tested the solution using Cluster.addResource() of the aws-eks module and would love to use cdk8s in order to provide some reusable constructs for our customers.

Describe the solution you’d like Here’s a sample of what we are thinking of building.

main.ts:

import cdk = require('@aws-cdk/core');
import k8s = require('cdk8s');
import eks = require('@aws-cdk/aws-eks');
import { SparkEksJob } from './spark-eks-job';

class SparkPiChart extends k8s.Chart  {
  constructor(scope: cdk.Construct, ns: string) {
    super(scope, ns);

    // my spark job image
    new SparkEksJob(this, 'SparkPi', {
      image: 'mysparkimage',
      command: [
         "/bin/sh",
          "-c",
          "/opt/spark/bin/spark-submit",
          "/opt/spark/examples/jars/spark-examples_2.11-2.4.4.jar"
       ]
    });
  }
}

const jobApp = new k8s.App();
const sparkPi = new SparkPiChart(app, 'spark-pi-job');

// This will be its own construct, but leaving this here for simplicity
const cluster = new eks.Cluster(this, 'job-cluster');

// add the chart 
cluster.addChart(sparkPi);

// synthesize
cdk.app.synth();

spark-eks-job.ts:

import { Construct } from '@aws-cdk/core';
import { App, Chart } from 'cdk8s';

// imported constructs
import { Job } from './imports/k8s';

export interface SparkEksJobOptions {
  /** 
    * The Docker image to use for this service.
  */
  readonly image: string;

  /**
   * Command to run
   */
  readonly command: Array<string>;
}

export class SparkEksJob extends Construct {
  constructor(scope: Construct, ns: string, options: SparkEksJobOptions) {
    super(scope, ns);
    
    // the rest of my k8s Job definition
    new Job(this, 'SparkJob', {
      ...
    });
  }
}

Describe alternatives you’ve considered AWS CDK + Helm charts

I can work with you on example implementation/testing.

[eladb]

@askulkarni2 I think the API you propose (eksCluster.addChart(cdk8sChart)) makes sense and as I said should be easy to implement with the existing eksCluster.addResource(manifest). Let’s start with this.

I was also thinking that we should consider a more “deep integration” between the two frameworks, so it will be possible to define constructs that include both AWS CDK resources and cdk8s resources and vend them as abstractions.

[eladb]

Use 0.16.0 for now. 0.17.0 is incompatible with CDK 1.30.0. Hopefully this will get resolved in CDK 1.31.0