Limit md5 import to allow for compatibility with fips environments#1288
Limit md5 import to allow for compatibility with fips environments#1288lratc wants to merge 3 commits intoapache:trunkfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the Cassandra driver’s MD5-based token hashing to explicitly mark MD5 usage as non-security-related, improving compatibility with FIPS-140 environments where MD5 may be disallowed for security purposes.
Changes:
- Replace
from hashlib import md5withimport hashlib. - Update
MD5Token.hash_fnto callhashlib.md5(..., usedforsecurity=False).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if isinstance(key, str): | ||
| key = key.encode('UTF-8') | ||
| return abs(varint_unpack(md5(key).digest())) | ||
| return abs(varint_unpack(hashlib.md5(key,usedforsecurity=False).digest())) |
There was a problem hiding this comment.
hashlib.md5(..., usedforsecurity=False) is not supported by all Python implementations (e.g., PyPy may raise TypeError: md5() takes no keyword arguments), which would break token hashing at runtime. Consider feature-detecting support once (e.g., try calling with usedforsecurity=False and fall back to a call without the kwarg) so the driver remains compatible while still fixing FIPS on CPython/OpenSSL builds that honor the flag.
There was a problem hiding this comment.
I was a bit concerned about this as well. The CPython issue in question (as well as the Python docs for the minimum support Python runtime) suggest that support for the usedforsecurity arg went in with Python 3.9. Assuming that's true we should be okay with all officially supported versions (since we only go back to 3.10)... but we would be introducing a breaking change for earlier Python versions. That's not critical since we don't claim support... but we also generally are pretty cautious about introducing changes known to break earlier versions.
Something like executing this functionality in a try block (and re-executing if we get the expected exception) might address this but... honestly, I'm a bit more worried about something else.
Those same 3.10 docs include the following:
md5() is normally available as well, though it may be missing or blocked if you are using a rare “FIPS compliant” build of Python.
That... isn't great. This potentially introduces a whole new class of errors; now we're not talking about the args md5() supports... instead we're talking about whether md5() exists at all.
I did some random Google searching to try to identify cases in which this might occur, especially since all of this functionality comes from OpenSSL (or an equivalent) in modern Python. Near as I can tell it requires some kind of custom build aimed at limiting exposed hash algorithms in order to meet the FIPS standard but nothing I found was super-clear on that point.
But since the whole point of this PR is better support for FIPS-compliant environments I'm starting to wonder if we shouldn't also handle the case in which md5() isn't present.
@bschoening any thoughts?
There was a problem hiding this comment.
@absurdfarce could move the import into Class MD5Token and only import if it's instantiated.
There was a problem hiding this comment.
You raise some very good points - on further inspection, the usedforsecurity=False isn't resolving the issue I was experiencing - not trying to import md5 at that point is what has resolved the issue. Moving the import into the class would be a much better solution.
| from collections.abc import Mapping | ||
| from functools import total_ordering | ||
| from hashlib import md5 | ||
| import hashlib |
There was a problem hiding this comment.
would prefer we keep the narrow import scope on a security librarry.
There was a problem hiding this comment.
Agreed. I have changed the approach to use a similar pattern to the one used for murmur3 (I inverted the if logic to avoid nesting the if statements).
| if isinstance(key, str): | ||
| key = key.encode('UTF-8') | ||
| return abs(varint_unpack(md5(key).digest())) | ||
| return abs(varint_unpack(hashlib.md5(key,usedforsecurity=False).digest())) |
There was a problem hiding this comment.
@absurdfarce could move the import into Class MD5Token and only import if it's instantiated.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@lratc @absurdfarce what about this approach: I don't think catching the exception is necessarily helpful. Note, this suggest reworking import murmur3 along the same lines. |
|
I feel like we need to have a larger conversation here. As I understand things Apache Cassandra itself is not compliant with FIPS 140. It sounds like there is a variant out there which claims to be but I haven't looked into it at all. The Python client also doesn't claim FIPS 140 compliance. I mention all of this mainly to say that I feel like any attempt at supporting FIPS environments should be treated as a first-class effort rather than as a single PR. It's also worth pointing out that the existing NoMurmur3 pattern may not be a good model to follow. The current driver code includes a Python impl of murmur3 which is used in case the C version isn't available for some reason (see this code for more on that). As a result cassandra.murmur3 will always be available which means this check is completely unnecessary. More generally I question whether throwing a NoMurmur3/NoMD5 exception is the right approach. While those errors are the root cause of the problem they happen pretty late in the process. It seems like a more informative exception for the user would be to throw an exception from the functionality that uses those digests saying that MD5 isn't available so no information can be computed. MD5 is used in Metadata.get_replicas() so it seems like the exception should be triggered there... but that entails knowing that MD5/Murmur3/other hash algorithm isn't available at that point in the process. Which in turn gets back to my original point about treating FIPS compliance as a top-level feature to be added rather than something we try to patch via a single PR. Thoughts? |
|
I agree that full FIPS 140 compliance would be a broader conversation if we had reason to tackle that. However, I believe we can make meaningful incremental improvements here without overcomplicating the driver's architecture. Specifically, adopting usedforsecurity=False is simply good practice for modern Python. Since this was introduced in Python 3.9, it serves as explicit documentation that our use of MD5 isn't for cryptographic security. This is particularly helpful for users running automated compliance scanners; it allows those tools to recognize the usage as a non-issue rather than flagging it as a vulnerability. Regarding the placement of the import: since the RandomPartitioner is rarely used in modern deployments, we could further "ringfence" the dependency by moving the MD5 import directly into the MD5Token class. This keeps the impact localized and ensures that the majority of users—who aren't using this partitioner—never even touch that logic and would never see an exception. We could remove the term "FIPS" from the Jira and just focus on conforming with Python 3.9+ security best practices. |
In a FIPS-140 environment, the cassandra-python-driver may error as md5 is not compiled into the hashlib module.
ModuleNotFoundError: No module named 'md5'As md5 is not available, the MD5Token class may not be used but the overall module is still usable.