Tensorboard can't work on Macos Catalina

Diagnostics output

--- check: autoidentify
INFO: diagnose_tensorboard.py version 27fd3ab75df3bb79262c4ef0b9bdf2b993092670

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=4, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Darwin', nodename='hong.local', release='19.0.0', version='Darwin Kernel Version 19.0.0: Thu Oct 17 16:17:15 PDT 2019; root:xnu-6153.41.3~29/RELEASE_X86_64', machine='x86_64')
INFO: sys.getwindowsversion(): N/A

--- check: package_management
INFO: has conda-meta: True
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==1.14.0
INFO: installed: tensorflow==1.14.0
INFO: installed: tensorflow-estimator==1.14.0

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '1.14.0'

--- check: tensorflow_python_version
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Fontconfig warning: ignoring UTF-8: not a valid region tag
INFO: Could not open font file /System/Library/Fonts/Apple Color Emoji.ttc: In FT2Font: Could not set the fontsize
INFO: Could not open font file /System/Library/Fonts/Supplemental/NISC18030.ttf: In FT2Font: Could not set the fontsize
INFO: generated new fontManager
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
INFO: tensorflow.__version__: '1.14.0'
INFO: tensorflow.__git_version__: 'unknown'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'/Users/fyf/anaconda3/anaconda3/bin/tensorboard\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 1024>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 1024>
Loopback infos: [(<AddressFamily.AF_INET6: 30>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET6: 30>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'hong.local'

--- check: stat_tensorboardinfo
INFO: directory: /var/folders/hy/v6wkpzjn3zg9tkzs14gmjg080000gq/T/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=8605918756, st_dev=16777220, st_nlink=2, st_uid=503, st_gid=20, st_size=64, st_atime=1574299215, st_mtime=1574321651, st_ctime=1574321651)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.8.1
alabaster==0.7.12
anaconda-client==1.7.2
anaconda-navigator==1.9.7
anaconda-project==0.8.3
appnope==0.1.0
appscript==1.0.1
asn1crypto==1.0.1
astor==0.8.0
astroid==2.3.1
astropy==3.2.2
atomicwrites==1.3.0
attrs==19.2.0
Babel==2.7.0
backcall==0.1.0
backports.functools-lru-cache==1.6.1
backports.os==0.1.1
backports.shutil-get-terminal-size==1.0.0
backports.tempfile==1.0
backports.weakref==1.0.post1
beautifulsoup4==4.8.0
bitarray==1.0.1
bkcharts==0.2
bleach==3.1.0
bokeh==1.3.4
boto==2.49.0
Bottleneck==1.2.1
certifi==2019.9.11
cffi==1.12.3
chardet==3.0.4
Click==7.0
cloudpickle==1.2.2
clyent==1.2.2
colorama==0.4.1
conda==4.7.12
conda-build==3.18.9
conda-package-handling==1.6.0
conda-verify==3.4.2
contextlib2==0.6.0
cryptography==2.7
cycler==0.10.0
Cython==0.29.13
cytoolz==0.10.0
dask==2.5.2
decorator==4.4.0
defusedxml==0.6.0
distributed==2.5.2
docutils==0.15.2
entrypoints==0.3
et-xmlfile==1.0.1
fastcache==1.1.0
filelock==3.0.12
Flask==1.1.1
fsspec==0.5.2
future==0.18.2
gast==0.3.2
gevent==1.4.0
glob2==0.7
gmpy2==2.0.8
greenlet==0.4.15
grpcio==1.16.1
h5py==2.9.0
HeapDict==1.0.1
html5lib==1.0.1
idna==2.8
imageio==2.6.0
imagesize==1.1.0
importlib-metadata==0.23
ipykernel==5.1.2
ipython==7.8.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
isort==4.3.21
itsdangerous==1.1.0
jdcal==1.4.1
jedi==0.15.1
Jinja2==2.10.3
joblib==0.13.2
json5==0.8.5
jsonschema==3.0.2
jupyter==1.0.0
jupyter-client==5.3.3
jupyter-console==6.0.0
jupyter-core==4.5.0
jupyterlab==1.1.4
jupyterlab-server==1.0.6
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
keyring==18.0.0
kiwisolver==1.1.0
lazy-object-proxy==1.4.2
libarchive-c==2.8
lief==0.9.0
llvmlite==0.29.0
locket==0.2.0
lxml==4.4.1
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.1
mccabe==0.6.1
mistune==0.8.4
mkl-fft==1.0.14
mkl-random==1.1.0
mkl-service==2.3.0
mock==3.0.5
more-itertools==7.2.0
mpmath==1.1.0
msgpack==0.6.1
multipledispatch==0.6.0
navigator-updater==0.2.1
nbconvert==5.6.0
nbformat==4.4.0
networkx==2.3
nltk==3.4.5
nose==1.3.7
notebook==6.0.1
numba==0.45.1
numexpr==2.7.0
numpy==1.17.2
numpydoc==0.9.1
olefile==0.46
openpyxl==3.0.0
packaging==19.2
pandas==0.25.1
pandocfilters==1.4.2
parso==0.5.1
partd==1.0.0
path.py==12.0.1
pathlib2==2.3.5
patsy==0.5.1
pep8==1.7.1
pexpect==4.7.0
pickleshare==0.7.5
Pillow==6.2.0
pip==19.2.3
pkginfo==1.5.0.1
pluggy==0.13.0
ply==3.11
prometheus-client==0.7.1
prompt-toolkit==2.0.10
protobuf==3.10.1
psutil==5.6.3
ptyprocess==0.6.0
py==1.8.0
pycodestyle==2.5.0
pycosat==0.6.3
pycparser==2.19
pycrypto==2.6.1
pycurl==7.43.0.3
pyflakes==2.1.1
Pygments==2.4.2
pylint==2.4.2
pyodbc==4.0.27
pyOpenSSL==19.0.0
pyparsing==2.4.2
pyrsistent==0.15.4
PySocks==1.7.1
pytest==5.2.1
pytest-arraydiff==0.3
pytest-astropy==0.5.0
pytest-doctestplus==0.4.0
pytest-openfiles==0.4.0
pytest-remotedata==0.3.2
python-dateutil==2.8.0
pytz==2019.3
PyWavelets==1.0.3
PyYAML==5.1.2
pyzmq==18.1.0
QtAwesome==0.6.0
qtconsole==4.5.5
QtPy==1.9.0
requests==2.22.0
rope==0.14.0
ruamel-yaml==0.15.46
scikit-image==0.15.0
scikit-learn==0.21.3
scipy==1.3.1
seaborn==0.9.0
Send2Trash==1.5.0
setuptools==41.4.0
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.12.0
snowballstemmer==2.0.0
sortedcollections==1.1.2
sortedcontainers==2.1.0
soupsieve==1.9.3
Sphinx==2.2.0
sphinxcontrib-applehelp==1.0.1
sphinxcontrib-devhelp==1.0.1
sphinxcontrib-htmlhelp==1.0.2
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.2
sphinxcontrib-serializinghtml==1.1.3
sphinxcontrib-websupport==1.1.2
spyder==3.3.6
spyder-kernels==0.5.2
SQLAlchemy==1.3.9
statsmodels==0.10.1
sympy==1.4
tables==3.5.2
tblib==1.4.0
tensorboard==1.14.0
tensorflow==1.14.0
tensorflow-estimator==1.14.0
termcolor==1.1.0
terminado==0.8.2
testpath==0.4.2
toolz==0.10.0
tornado==6.0.3
tqdm==4.36.1
traitlets==4.3.3
unicodecsv==0.14.1
urllib3==1.24.2
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.16.0
wheel==0.33.6
widgetsnbextension==3.5.1
wrapt==1.11.2
wurlitzer==1.0.3
xlrd==1.2.0
XlsxWriter==1.2.1
xlwings==0.15.10
xlwt==1.3.0
zict==1.0.0
zipp==0.6.0

After I update my Mac to Catalina 10.15.1, my anaconda was broke. I carefully uninstalled anaconda by the following code.

conda install anaconda-clean 
anaconda-clean --yes           
rm -rf ~/anaconda3             
rm -rf ~/.anaconda_backup      

After that I reinstalled anaconda 4.7.12. Then I installed tensorboard and TensorFlow by default latest version. I run tensorboard in my terminal. It shows:

[libprotobuf ERROR external/protobuf_archive/src/google/protobuf/descriptor_database.cc:334] Invalid file descriptor data passed to EncodedDescriptorDatabase::Add().
[libprotobuf FATAL external/protobuf_archive/src/google/protobuf/descriptor.cc:1370] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
Abort trap: 6

I reinstalled tensorboard and TensorFlow both by version 1.14, and it tried to run tensorboard again. It shows:

/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/Users/fyf/anaconda3/anaconda3/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
[libprotobuf ERROR external/protobuf_archive/src/google/protobuf/descriptor_database.cc:334] Invalid file descriptor data passed to EncodedDescriptorDatabase::Add().
[libprotobuf FATAL external/protobuf_archive/src/google/protobuf/descriptor.cc:1370] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
Abort trap: 6

Thank you for your patience. All the best with your work.

[jasonjanofsky]

We have seen this as well and rebuilt a machine over it before deciding to dig in. Here is what we found, after reading this article. We attempted the reinstall, from above of tensorflow and protobuf. This, however, didn’t fix the problem.

It appears that tensorflow, by default, when pinned to 1.14 installs protobuf > 3.8.0. We are also using the google-cloud-storage sdk which needed to be reinstalled after reinstalling tensorflow and protobuf. We tried flipping the order and uninstalled tensorflow and protobuf then reinstalled google-cloud-storage and, finally, installed tensorflow by itself. Upon doing this, the problem was fixed.

Here is our logic: The google-cloud-storage sdk installed protobuf 3.8.0 as a dependency. when we installed tensorflow, it found this to be acceptable and didn’t touch protobuf. When we installed tensorflow first, it installed protobuf 3.10 or greater and google-cloud-storage found this to be acceptable. Whenever protobuf > than 3.10 is installed on a Catalina machine, the protobuf error occurs.

So, one might fix this issue by installing protobuf 3.8 or validating that below 3.10 is installed.

Our environment is a little different in that we use conda but the same general theory applies. Finally, we were able to replicate this on two machines along w/the fix.

We also tracked this down as far as we could into the python internals: In the file: imp.py _load(spec)

Attempting to load the file _pywrap_tensorflow_internal.so explodes the python interpreter.

Good luck!

[jhrcook]

We have seen this as well and rebuilt a machine over it before deciding to dig in. Here is what we found, after reading this article. We attempted the reinstall, from above of tensorflow and protobuf. This, however, didn’t fix the problem.

It appears that tensorflow, by default, when pinned to 1.14 installs protobuf > 3.8.0. We are also using the google-cloud-storage sdk which needed to be reinstalled after reinstalling tensorflow and protobuf. We tried flipping the order and uninstalled tensorflow and protobuf then reinstalled google-cloud-storage and, finally, installed tensorflow by itself. Upon doing this, the problem was fixed.

Here is our logic: The google-cloud-storage sdk installed protobuf 3.8.0 as a dependency. when we installed tensorflow, it found this to be acceptable and didn’t touch protobuf. When we installed tensorflow first, it installed protobuf 3.10 or greater and google-cloud-storage found this to be acceptable. Whenever protobuf > than 3.10 is installed on a Catalina machine, the protobuf error occurs.

So, one might fix this issue by installing protobuf 3.8 or validating that below 3.10 is installed.

Our environment is a little different in that we use conda but the same general theory applies. Finally, we were able to replicate this on two machines along w/the fix.

We also tracked this down as far as we could into the python internals: In the file: imp.py _load(spec)

Attempting to load the file _pywrap_tensorflow_internal.so explodes the python interpreter.

Good luck!

Uinstalling protobuf and tensorflow and reinstalling with

conda install protobuf=3.8
conda install tensorflow

worked to get TensorBoard working.