[Suggestion] A hook in aws-eks for cdk8s Chart
See original GitHub issueIs your feature request related to a problem? Please describe.
We would like to abstract the details of provisioning AWS infrastructure (EKS, etc.) from our customers (data scientists). They provide a Spark docker container with the application code and we run it for them. We are leveraging a new capability in Apache Spark so that we can run the job on EKS. We have tested the solution using Cluster.addResource()
of the aws-eks
module and would love to use cdk8s
in order to provide some reusable constructs for our customers.
Describe the solution you’d like Here’s a sample of what we are thinking of building.
main.ts
:
import cdk = require('@aws-cdk/core');
import k8s = require('cdk8s');
import eks = require('@aws-cdk/aws-eks');
import { SparkEksJob } from './spark-eks-job';
class SparkPiChart extends k8s.Chart {
constructor(scope: cdk.Construct, ns: string) {
super(scope, ns);
// my spark job image
new SparkEksJob(this, 'SparkPi', {
image: 'mysparkimage',
command: [
"/bin/sh",
"-c",
"/opt/spark/bin/spark-submit",
"/opt/spark/examples/jars/spark-examples_2.11-2.4.4.jar"
]
});
}
}
const jobApp = new k8s.App();
const sparkPi = new SparkPiChart(app, 'spark-pi-job');
// This will be its own construct, but leaving this here for simplicity
const cluster = new eks.Cluster(this, 'job-cluster');
// add the chart
cluster.addChart(sparkPi);
// synthesize
cdk.app.synth();
spark-eks-job.ts
:
import { Construct } from '@aws-cdk/core';
import { App, Chart } from 'cdk8s';
// imported constructs
import { Job } from './imports/k8s';
export interface SparkEksJobOptions {
/**
* The Docker image to use for this service.
*/
readonly image: string;
/**
* Command to run
*/
readonly command: Array<string>;
}
export class SparkEksJob extends Construct {
constructor(scope: Construct, ns: string, options: SparkEksJobOptions) {
super(scope, ns);
// the rest of my k8s Job definition
new Job(this, 'SparkJob', {
...
});
}
}
Describe alternatives you’ve considered AWS CDK + Helm charts
I can work with you on example implementation/testing.
Issue Analytics
- State:
- Created 4 years ago
- Comments:13 (8 by maintainers)
@askulkarni2 I think the API you propose (
eksCluster.addChart(cdk8sChart)
) makes sense and as I said should be easy to implement with the existingeksCluster.addResource(manifest)
. Let’s start with this.I was also thinking that we should consider a more “deep integration” between the two frameworks, so it will be possible to define constructs that include both AWS CDK resources and cdk8s resources and vend them as abstractions.
Use 0.16.0 for now. 0.17.0 is incompatible with CDK 1.30.0. Hopefully this will get resolved in CDK 1.31.0