Backing up binary data (Written by Hashicorp Vault) fails

I’m getting the following when attempting to run a consulate backup on my consul cluster:

(con)[Tue Nov 03 17:25:50] ~/r/temp/consulate$ consulate --api-host 10.1.15.125 --api-port 8500 kv backup --base64 -f consul_backup.json
Traceback (most recent call last):
  File "/Users/neil1/virtualenvs/con/bin/consulate", line 9, in <module>
    load_entry_point('consulate==0.6.0', 'console_scripts', 'consulate')()
  File "build/bdist.macosx-10.11-intel/egg/consulate/cli.py", line 469, in main
  File "build/bdist.macosx-10.11-intel/egg/consulate/cli.py", line 214, in kv_backup
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py", line 53, in b64encode
    encoded = binascii.b2a_base64(s)[:-1]
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2014' in position 562: ordinal not in range(128)    

Looks like https://github.com/gmr/consulate/issues/41

Python 2.7.10 Consulate 0.6.0 (Both pip and master)

We’ve tracked the issue to Hashicorp Vault writing binary data.

Using python3.5 we instead get the error:

(test)[Tue Nov 03 17:41:37] ~/r/temp$ consulate --api-host 10.1.15.125 --api-port 8500 kv backup --base64 -f consul_backup.json
Traceback (most recent call last):
  File "/usr/local/bin/consulate", line 11, in <module>
    sys.exit(main())
  File "/Library/Python/2.7/site-packages/consulate/cli.py", line 469, in main
    KV_ACTIONS[args.action](consul, args)
  File "/Library/Python/2.7/site-packages/consulate/cli.py", line 214, in kv_backup
    records = [(k, f, base64.b64encode(v)) for k,f,v in records]
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.py", line 53, in b64encode
    encoded = binascii.b2a_base64(s)[:-1]
TypeError: must be string or buffer, not None

If there’s any other information we can provide please let us know!

[alexku-o]

@tyx thanks. I actually tried restore with -b earlier but it was giving me some error. I ended up just using consul-backup because we have a simple backup/restore use case and it works perfectly well for us (and the fact that our test backup/restore worked with consul-backup but not with consulate).

edit: In case it came across that way, my comment here was certainly not in any way to discredit the incredible work that’s been done on consulate. I appreciate a great deal of the work here, and am thankful.

[alexku-o]

@ben-nz - using -b like consulate kv backup -b worked for me (following what was suggested in https://github.com/gmr/consulate/issues/41)