question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ETag checksum fails on FIPS-enabled systems when using MD5

See original GitHub issue

Checklist

  • The bug is reproducible against the latest release and/or master.
  • There are no similar issues or pull requests to fix it yet.

Describe the bug

The ETag checksum fails when using MD5. This is causing Starlette to not work at all under Red Hat Enterprise Linux when FIPS mode is enabled.

Debugging material

Here’s the exception that’s thrown:

INFO:     10.42.1.7:34422 - "GET /app/static/foo.html HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 656, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 408, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/staticfiles.py", line 97, in __call__
    response = await self.get_response(path, scope)
  File "/usr/local/lib/python3.8/site-packages/starlette/staticfiles.py", line 118, in get_response
    return self.file_response(full_path, stat_result, scope)
  File "/usr/local/lib/python3.8/site-packages/starlette/staticfiles.py", line 173, in file_response
    response = FileResponse(
  File "/usr/local/lib/python3.8/site-packages/starlette/responses.py", line 267, in __init__
    self.set_stat_headers(stat_result)
  File "/usr/local/lib/python3.8/site-packages/starlette/responses.py", line 273, in set_stat_headers
    etag = hashlib.md5(etag_base.encode()).hexdigest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

Environment

  • OS: Red Hat Enterprise Linux 8 in FIPS mode
  • Python version: 3.8.8
  • Starlette version: 0.16.0

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:13 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
landtunacommented, Jan 13, 2022

Confirmed that version 0aef1724cfafbe23f846979d427a5a173667f6b7 with usedforsecurity=False in _compat.py works on the FIPS system. I don’t think I can check master (after your merge) because it doesn’t work for me with the latest release of FastAPI. But I think we’re good to go now with regards to _compat.py!

1reaction
landtunacommented, Dec 17, 2021

@tomchristie I think that’s not quite accurate. I believe RHEL was the first implementer and backported this feature, and it’s on the RHEL family where everything has this problem. I know it’s in some of the RHEL Python 2.7 libs, but I’m not positive about the others.

Since the implementation didn’t check the Python version but did use try/except, I think this fix works for everyone who needs it to to work.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Aws S3 etag not matching md5 after KMS encryption
In AWS S3 the etag is not an MD5 checksum. If just happens that this is the case in the past but AWS...
Read more >
Hashes and ETags: best practices | Cloud Storage
This page describes best practices for performing validations using either CRC32C or MD5 checksums. Protect against data corruption by using hashes. There are...
Read more >
Check the integrity of an object uploaded to Amazon S3
Note: When you use the Content-MD5 header, Amazon S3 checks the object against the provided Content-MD5 value. If the values do not match,...
Read more >
Building scalable checksums | AWS Media Blog
Performing a checksum on a file entails using an algorithm to iterate ... MD5 digests are used to determine the ETag for the...
Read more >
etag - Go Packages
In general, an S3 ETag is an MD5 checksum of the object content. ... for S3 clients when an object is encrypted using...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found