Boto3 client error in multi threaded environment.

I am using uwsgi behind django in production with threads enabled. I occasionally get

KeyError
'endpoint_resolver'

from this line django_s3_storage/storage.py in _setup at line 140

        if self.settings.AWS_S3_ENDPOINT_URL:
            connection_kwargs["endpoint_url"] = self.settings.AWS_S3_ENDPOINT_URL
        self.s3_connection = boto3.client("s3", config=Config(
            s3={"addressing_style": self.settings.AWS_S3_ADDRESSING_STYLE},
            signature_version=self.settings.AWS_S3_SIGNATURE_VERSION,
        ), **connection_kwargs)
    def _setting_changed_received(self, setting, **kwargs):
        if setting.startswith("AWS_"):
            # HACK: No supported way to close the HTTP session from boto3... :S
            self.s3_connection._endpoint.http_session.close()

upon searching I got this from boto3 documentation. Is it possible to do the same? like django-storages does here.

[etianen]

I’ve made your suggested fix to the master branch. Please let me know if it works. 😃

As a general comment on the boto3 library though - what total twaddle is this?! (I know it’s not your library, but I’m venting). Sure… making the client not thread-safe, I understand that not everything should be thread-safe. But If I explicitly create a new client in a new thread, why make it implicitly share a non-thread-safe data structure?! That’s the worst of all worlds!

The boto3 library has been a pain to transition to. All the return values are untyped dicts. The documentation is all over the place. I don’t see how it can be remotely considered an upgrade from boto2. The only reason to transition to is was that the boto2 support was being dropped.

On 10 January 2018 at 10:18, Noorhteen Raja J [e-mail] wrote:

Reopened #69 https://github.com/etianen/django-s3-storage/issues/69.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/etianen/django-s3-storage/issues/69#event-1417352325, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJFCJ21rNS7G9WlIlZ7dLYB4RmPM7F_ks5tJI5lgaJpZM4RVQ8- .

[etianen]

0.12.3 is now released. Thanks for confirming the fix!

On 11 January 2018 at 10:46, Noorhteen Raja J [e-mail] wrote:

It works now. Thanks for the quick change 👍

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/etianen/django-s3-storage/issues/69#issuecomment-356897107, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJFCDxFKd3QEyB5V48LNul6cp7bFkcDks5tJeZ4gaJpZM4RVQ8- .