ETag checksum fails on FIPS-enabled systems when using MD5
See original GitHub issueChecklist
- The bug is reproducible against the latest release and/or
master
. - There are no similar issues or pull requests to fix it yet.
Describe the bug
The ETag checksum fails when using MD5. This is causing Starlette to not work at all under Red Hat Enterprise Linux when FIPS mode is enabled.
Debugging material
Here’s the exception that’s thrown:
INFO: 10.42.1.7:34422 - "GET /app/static/foo.html HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi
result = await app(self.scope, self.receive, self.send)
File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 208, in __call__
await super().__call__(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
raise exc
File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
raise exc
File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 656, in __call__
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 408, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.8/site-packages/starlette/staticfiles.py", line 97, in __call__
response = await self.get_response(path, scope)
File "/usr/local/lib/python3.8/site-packages/starlette/staticfiles.py", line 118, in get_response
return self.file_response(full_path, stat_result, scope)
File "/usr/local/lib/python3.8/site-packages/starlette/staticfiles.py", line 173, in file_response
response = FileResponse(
File "/usr/local/lib/python3.8/site-packages/starlette/responses.py", line 267, in __init__
self.set_stat_headers(stat_result)
File "/usr/local/lib/python3.8/site-packages/starlette/responses.py", line 273, in set_stat_headers
etag = hashlib.md5(etag_base.encode()).hexdigest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
Environment
- OS: Red Hat Enterprise Linux 8 in FIPS mode
- Python version: 3.8.8
- Starlette version: 0.16.0
Issue Analytics
- State:
- Created 2 years ago
- Comments:13 (8 by maintainers)
Top Results From Across the Web
Aws S3 etag not matching md5 after KMS encryption
In AWS S3 the etag is not an MD5 checksum. If just happens that this is the case in the past but AWS...
Read more >Hashes and ETags: best practices | Cloud Storage
This page describes best practices for performing validations using either CRC32C or MD5 checksums. Protect against data corruption by using hashes. There are...
Read more >Check the integrity of an object uploaded to Amazon S3
Note: When you use the Content-MD5 header, Amazon S3 checks the object against the provided Content-MD5 value. If the values do not match,...
Read more >Building scalable checksums | AWS Media Blog
Performing a checksum on a file entails using an algorithm to iterate ... MD5 digests are used to determine the ETag for the...
Read more >etag - Go Packages
In general, an S3 ETag is an MD5 checksum of the object content. ... for S3 clients when an object is encrypted using...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Confirmed that version 0aef1724cfafbe23f846979d427a5a173667f6b7 with
usedforsecurity=False
in_compat.py
works on the FIPS system. I don’t think I can check master (after your merge) because it doesn’t work for me with the latest release of FastAPI. But I think we’re good to go now with regards to_compat.py
!@tomchristie I think that’s not quite accurate. I believe RHEL was the first implementer and backported this feature, and it’s on the RHEL family where everything has this problem. I know it’s in some of the RHEL Python 2.7 libs, but I’m not positive about the others.
Since the implementation didn’t check the Python version but did use try/except, I think this fix works for everyone who needs it to to work.