question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`ignore` doesn't seem to work in more complicated setup (Django + S3 / boto)

See original GitHub issue

I am using Django with S3 as the file storage (which uses boto3/botocore libs). I have a “management” command which creates some Django instances - I use freezegun, so these instances appear to be created in the past. However some of these models contain files, that are being saved to S3 - which raises some exceptions:

...
  File "/home/rado/.virtualenvs/twisto/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/rado/.virtualenvs/twisto/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (RequestTimeTooSkewed) when calling the ListObjectsV2 operation: The difference between the request time and the current time is too large.

this is due to some authentication code in botocore.auth module, that checks for time differences. I tried to solve this issue using ignore parameter (freeze_time(datetime_in_past, ignore=['botocore'])), but it didn’t help, botocore.auth still used the FakeDatetime (I used debugger with breakpoints to pinpoint the issue).

I tried to recreate the issue with a simple reproducible example, so I came up with following:

# test.py
from freezegun import freeze_time
from datetime import datetime
import stuff_in_between

import test_time #  <-- note this unused import

with freeze_time(datetime(2019, 1, 1, 1, 1, 1, 1), ignore=['test_time']):
    print(f'frozen time: {datetime.now()}')
    stuff_in_between.call_test_time()
# stuff_in_between.py
def call_test_time():
    import test_time
    test_time.test_time()
# test_time.py
from datetime import datetime

def test_time():
    print(f'real datetime.now: {datetime.now()}')

This code correctly works, freeze_time ignore test_time and it prints real datetime.now: <real time>. However if I omit the marked unused import in test.py, it doesn’t work - freeze_time doesn’t ignore the module and the script prints out real datetime.now: <fake time>.

I thought I could try out something similar in my real Django go, so I did following:

...
import botocore
import botocore.auth
# hell, try to import everything possible related to S3
with freeze_time(datetime_in_past, ignore=['botocore']:
   ...
   # create some instances, that also save files to S3

This hasn’t worked no matter what I imported or ignored, botocore.auth still continued to use FakeDatetime… Is there a bug in freeze_time? Or am I misusing the library in some way? Why does it behave like this?

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:8

github_iconTop GitHub Comments

2reactions
adrianregocommented, Feb 19, 2021

I believe we’ve run into a similar issue with urllib3 since a recent change:

https://github.com/urllib3/urllib3/commit/3f21165969b838fda29898cbd7218ac9578e319b#diff-542c95910a277028550d2c8943e8c49bbcb10f9af96e35d0a0eee99b8cfe9e8c

Since then our tests have been printing SystemTimeWarnings and adding both requests and urllib3 to the ignore list doesn’t have an effect.

1reaction
MythicManiaccommented, Mar 27, 2022

Also ran into this issue, specifically getting

botocore.exceptions.ClientError: An error occurred (RequestTimeTooSkewed) when calling the PutObject operation: The diff
erence between the request time and the server's time is too large.

when writing tests that also happen to use s3 based storage. Adding the related packages to the ignore list doesn’t seem to have any effect.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python - freeze_time `ignore` doesn't seem to work in more ...
I am using Django with S3 as the file storage (which uses boto3/botocore libs). I have a "management" command which creates some Django...
Read more >
Top 10 Mistakes That Django Developers Make - Toptal
In this tutorial, we will look at some common mistakes that are often made by Django developers and ways to avoid them. Whether...
Read more >
Amazon S3 — django-storages 1.13.1 documentation
The AWS profile to use instead of AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY . All configuration information other than the key id and secret key is...
Read more >
How to Upload Files to S3 Using Django Storages - YouTube
In this video I'll talk about how to upload your media files to Amazon S3 using the Django storages library.Need one-on-one help with...
Read more >
How to Setup Amazon S3 in a Django Project
You will need to install two Python libraries: ... The boto3 library is a public API client to access the Amazon Web Services...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found