(aws-eks): Construct Library custom resources doesn't use proxy properly
See original GitHub issueI’m trying to test new feature related to provision within the VPC lambda functions related to EKS configuration like ClusterHandler
. Lambdas are placed to vpc and it’s great. But I got error connecting to EKS api via proxy
Reproduction Steps
self.cluster = aws_eks.Cluster(
scope=self,
id='cluster',
cluster_name="cluster-" + environment,
endpoint_access=aws_eks.EndpointAccess.PUBLIC_AND_PRIVATE,
default_capacity=0,
vpc=vpc,
vpc_subnets=[aws_ec2.SubnetSelection(subnets=[subnet_a_eks, subnet_b_eks, subnet_c_eks])],
# issue with 3 subnet
place_cluster_handler_in_vpc=True,
version=cluster_version,
cluster_handler_environment={
"http_proxy": "http://login:pass@proxy.cloud.local:8080/"
},
kubectl_environment={
"http_proxy": "http://login:pass@proxy.cloud.local:8080/"
},
security_group=eks_control_plane_sg,
role=eks_control_plane_role,
)
What did you expect to happen?
successful cluster creation
What actually happened?
Cloudwatch log of ProviderframeworkonEvent. function:
2021-01-09T14:37:27.905Z e785fa78-c5f8-471c-a495-59d75389a6c6 INFO [provider-framework] submit response to cloudformation { "Status": "FAILED", "Reason": "Error: connect ETIMEDOUT 63.32.73.253:443\n at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1107:14)", "StackId": "arn:aws:cloudformation:eu-west-1:accountid:stack/eks-stack-develop-cdk/276db000-5285-11eb-ab35-0615947f7f49", "RequestId": "ce8a03d7-fdf3-4def-acb4-6219fb352732", "PhysicalResourceId": "AWSCDK::CustomResourceProviderFramework::CREATE_FAILED", "LogicalResourceId": "clusterC5B25D0D" }
Proxy is fine i tested it against same endpoint ( strange that it calls ec2 and nor eks api
[ec2-user@ip-10-60-233-255 ~]$ curl -vk https://ec2-63-32-73-253.eu-west-1.compute.amazonaws.com.
* Rebuilt URL to: https://ec2-63-32-73-253.eu-west-1.compute.amazonaws.com./
* Uses proxy env variable https_proxy == 'http://user:password@proxy.cloud.local:8080/'
* Trying 10.60.249.170...
* TCP_NODELAY set
* Connected to proxy.cloud.local (10.60.249.170) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to ec2-63-32-73-253.eu-west-1.compute.amazonaws.com:443
* Proxy auth using Basic with user 'user'
> CONNECT ec2-63-32-73-253.eu-west-1.compute.amazonaws.com:443 HTTP/1.1
> Host: ec2-63-32-73-253.eu-west-1.compute.amazonaws.com:443
> Proxy-Authorization: Basic token
> User-Agent: curl/7.61.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=lambda.eu-west-1.amazonaws.com
* start date: Dec 23 00:00:00 2020 GMT
* expire date: Jan 21 23:59:59 2022 GMT
* issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x17f8190)
> GET / HTTP/2
> Host: ec2-63-32-73-253.eu-west-1.compute.amazonaws.com
> User-Agent: curl/7.61.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 403
< date: Tue, 12 Jan 2021 13:31:59 GMT
< content-length: 127
< x-amzn-requestid: c6c6badc-56de-4e6a-8266-0d7971505c84
<
<MissingAuthenticationTokenException>
<Message>Missing Authentication Token</Message>
</MissingAuthenticationTokenException>
* Connection #0 to host proxy.cloud.local left intact
Environment
- **CDK CLI Version :1.83
- Framework Version:
- **Node.js Version: v14.13.0
- **OS :macOS
- **Language (Version): Python (3.9)
Other
I noticed that 5 lambda functions are created but only 1 of them “OnEventHandler” receives proxy configuration, but looks like it only one which interacts with api
This is 🐛 Bug Report
Issue Analytics
- State:
- Created 3 years ago
- Reactions:11
- Comments:12 (5 by maintainers)
Top Results From Across the Web
aws-cdk/aws-eks module - AWS Documentation
By default, this library will allocate a managed node group with 2 m5.large instances (this instance type suits most common use-cases, and is...
Read more >Amazon EKS add-ons: Advanced configuration | Containers
Given an existing Amazon EKS cluster, we want to use the AWS CLI to add the VPC CNI add-on to the existing cluster...
Read more >aws-cdk/custom-resources module - AWS Documentation
AWS CloudFormation custom resources are extension points to the provisioning engine. When CloudFormation needs to create, update or delete a custom resource ......
Read more >Amazon EKS troubleshooting - AWS Documentation
This chapter covers some common errors that you may see while using Amazon EKS and how to work around them. If you need...
Read more >Amazon EKS Construct Library - AWS Documentation
At the moment, the AWS::EKS::Cluster AWS CloudFormation resource does not support this behavior, so in order to support “programmatic kubectl”, such as applying ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@alexey-boyko
I guess that makes in case the proxy isn’t being applied correctly. I did a little digging and it seems that unlike python, nodejs doesn’t use any global env variables to configures proxy passes. Looks like every http client implements this independently.
We might need to do add explicit support for this in the library. I’ll try adding some more information soon.
⚠️COMMENT VISIBILITY WARNING⚠️
Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.