question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How can I set s3 connection to keep alive?

See original GitHub issue

I wanna uploading lots of files to s3 service every time. I made 10 workers to upload files using thread. But a message, “Resetting dropped connection”, is shown repeatedly. I guess s3 connection is newly opened per one time. How can I set connection to keep alive?

import threading
import boto3

s3 = boto3.resource('s3')

class Worker(threading.Thread):
    ...
    def run(self):
        s3.Object(bucket, key).put(Body=body)

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

5reactions
JordonPhillipscommented, Oct 6, 2016

The session itself isn’t thread safe, you’ll need to create a new one for each thread:

import threading
from boto3.session import Session


class Worker(threading.Thread):
    ...
    def run(self):
        s = Session()
        s3 = s.resource('s3')
1reaction
jaredtkatzcommented, Mar 22, 2018

What’s the difference between

s = Session()
s3 = s.resource('s3')

and s3 = boto3.resource('s3') ?

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to enable Keep Alive connection in AWS S3 or CloudFront?
Keep -alive connections are the norm -- they allow the browser to send the next request without the overhead of a new TCP...
Read more >
How to set Keep-Alive to the Header to request to S3? #829
Hi @skyleelove, by default the SDK will use connection pooling and keep alive for all requests made to S3. These connections may be...
Read more >
HTTP keep-alive is on by default in modular AWS SDK for ...
In v3, we keep the Node.js HTTP connections alive by default. ... In AWS SDK for JavaScript v2, the keepAlive option was not...
Read more >
S3 idle connection timeout | AWS re:Post
You can't control the connection timeout; but you might be able to keep the connection alive by sending TCP keepalives or other requests...
Read more >
Why does Amazon recommends reading 'all data' as soon as ...
Therefore looking for some input from the community. Thanks. amazon-s3 · slow-connection · keep-alive · Share.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found