Python 3 Port - TypeError: Object of type bytes is not JSON serializable

I am running into the following issue using Python 3.6 and Python 3.7 (I thought 3.7 might have resolved this)

2019-04-18 14:02:17.059 - keylime.cloudverifier - INFO - Starting Cloud Verifier (tornado) on port 8881, use <Ctrl-C> to stop
2019-04-18 14:02:17.059 - keylime.cloudverifier_common - INFO - Setting up TLS...
2019-04-18 14:02:17.059 - keylime.cloudverifier_common - INFO - Generating a new CA in /var/lib/keylime/cv_ca and a client certificate for connecting
2019-04-18 14:02:17.060 - keylime.cloudverifier_common - INFO - use keylime_ca -d /var/lib/keylime/cv_ca to manage this CA
2019-04-18 14:02:17.060 - keylime.cloudverifier_common - WARNING - CAUTION: using default password for CA, please set private_key_pw to a strong password
Traceback (most recent call last):
  File "/usr/local/bin/keylime_verifier", line 11, in <module>
    load_entry_point('keylime==1.2', 'console_scripts', 'keylime_verifier')()
  File "/usr/local/lib/python3.7/site-packages/keylime-1.2-py3.7.egg/keylime/cloud_verifier_tornado.py", line 495, in main
    context = cloud_verifier_common.init_mtls()
  File "/usr/local/lib/python3.7/site-packages/keylime-1.2-py3.7.egg/keylime/cloud_verifier_common.py", line 122, in init_mtls
    ca_util.cmd_init(tls_dir)
  File "/usr/local/lib/python3.7/site-packages/keylime-1.2-py3.7.egg/keylime/ca_util.py", line 153, in cmd_init
    write_private(priv)
  File "/usr/local/lib/python3.7/site-packages/keylime-1.2-py3.7.egg/keylime/ca_util.py", line 423, in write_private
    priv_encoded = json.dumps(priv)
  File "/usr/lib64/python3.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib64/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib64/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

Set up steps:

1. Install dependencies:

# dnf install python3-devel python3-pip python3-setuptools python3-tornado python3-virtualenv python3-zmq python3-yaml python3-m2crypto python3-pycryptodomex libselinux-python3
# dnf install procps libtool gcc make automake m2crypto redhat-rpm-config libselinux-python gnulib glib2-devel glib2-static uthash-devel wget which

1. Run 2to3 pm code base
2. Install keylime python3 setup.py install
3. Start the keylime_verifier

From what I can tell the issue starts here:

https://github.com/keylime/keylime/blob/e7c153a16fa47e3b411ddbcd18f04a66b37e8898/keylime/ca_util.py#L448

This takes the return from crypto.generate_random_key, encodes it as base64 and attempts to load that into a JSON object, this is then refused with TypeError: Object of type bytes is not JSON serializable

Opening this issue, not as a bug, but a means to discuss a way forwards.

Tagging @jetwhiz

[jetwhiz]

Hi @lukehinds

It looks like py3 now exports byte arrays from base64.b64encode instead of ASCII strings when performing its base64 encoding. We will need to convert this byte array back to ASCII for this to be JSON serializeable, using .decode('ascii').

For https://github.com/keylime/keylime/blob/e7c153a16fa47e3b411ddbcd18f04a66b37e8898/keylime/ca_util.py#L448

Change base64.b64encode(crypto.generate_random_key()) to base64.b64encode(crypto.generate_random_key()).decode('ascii') so it can be serialized.

Can you see if that fixes the issue for you? We might need to do this everywhere base64 is performed …

[jetwhiz]

@lukehinds – can you try bytes(nonce, encoding="utf8").hex() and see if that works?