question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pyathena encryption with kms key is not working

See original GitHub issue

I am querying Athena using PyAthena-1.4.6.

connection = connect(
    s3_staging_dir='s3://' + athena_results_s3_bucket + '/athena_results',
    region_name=region,
    encryption_option='SSE-KMS',
    kms_key=kms_key
)
cursor = connection.cursor()
cursor.execute(sql_query)

I am able to query the data, but the data in s3 staging_dir is not encrypted. I tried all possible values for ‘encryption_opion’. I could not find any documentation for this. `

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
chgangarajucommented, Feb 28, 2019

This following code is working for me

client = boto3.client('athena')
response = client.start_query_execution(
    QueryString=sql_query,
    QueryExecutionContext={
        'Database': database
    },
    ResultConfiguration={
        'OutputLocation': s3_staging_dir,
        'EncryptionConfiguration': {
            'EncryptionOption': 'SSE_KMS',
            'KmsKey': kms_key_arn
        }
    }
)
0reactions
chgangarajucommented, Feb 28, 2019

It is working as expected with PyAthena also.

connection = connect(
    s3_staging_dir='s3://' + athena_results_s3_bucket + '/athena_results',
    region_name=region,
    encryption_option='SSE_KMS',
    kms_key=kms_key
)
cursor = connection.cursor()
cursor.execute(sql_query)

image

But the results inside s3_staging_dir is not encrypted (With PyAthena and boto3) . But it is actualy encrypted at s3://aws-athena-query-results-<account_id>-<region>/Unsaved/2019/02/28/0a7d28b4-d785-41e7-b0bd-40fc5a87c05e.csv

Is there any way to encrypt this as well?

boto3 issue reference

Read more comments on GitHub >

github_iconTop Results From Across the Web

Troubleshooting in Athena - AWS Documentation
The Athena team has gathered the following troubleshooting information from customer issues. Although not comprehensive, it includes advice regarding some ...
Read more >
Athena — Boto3 Docs 1.26.33 documentation - Amazon AWS
It specifies whether query results must be encrypted, for all queries that run in this workgroup. For SSE_KMS and CSE_KMS , this is...
Read more >
AccessDenied on AWS Athena - Stack Overflow
@JohnRotenstein I have the same problem but in my case I do set the acl to bucket-owner-full-control when copying object between accounts ...
Read more >
Amazon S3 and Amazon Athena Encryption - Cloud Academy
Amazon S3 and Amazon Athena Encryption - AWS Encryption for Data Analytics ... So, to be clear, the KMS CMK is used to...
Read more >
How to use the pyathena.error.DataError function in pyathena | Snyk
Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found