PR Curve Summary Ops Not Working with TensorFlow 2.0

Environment information (required)

Diagnostics

Diagnostics output

--- check: autoidentify
INFO: diagnose_tensorboard.py version 4725c70c7ed724e2d1b9ba5618d7c30b957ee8a4

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=4, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='[redacted]', release='3.10.0-1062.1.1.el7.x86_64', version='#1 SMP Fri Sep 13 22:55:44 UTC 2019', machine='x86_64')
INFO: sys.getwindowsversion(): N/A

--- check: package_management
INFO: has conda-meta: True
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==2.0.1
INFO: installed: tensorflow-gpu==2.0.0
INFO: installed: tensorflow-estimator==2.0.1

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.0.1'

--- check: tensorflow_python_version
INFO: tensorflow.__version__: '2.0.0'
INFO: tensorflow.__git_version__: 'v2.0.0-rc2-26-g64c3d38'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'[redacted]\n'

--- check: readable_fqdn
INFO: socket.getfqdn(): 'gpu-1'

--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=262158, st_dev=64772, st_nlink=2, st_uid=1002, st_gid=1003, st_size=4096, st_atime=1572522005, st_mtime=1573152354, st_ctime=1573152354)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['[redacted]/.conda/envs/[redacted]/lib/python3.7/site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.8.0
ana==0.5
archinfo==8.19.7.25
asn1crypto==1.2.0
astor==0.8.0
atomicwrites==1.3.0
attrs==19.3.0
backcall==0.1.0
bitstring==3.1.5
bleach==3.1.0
boto==2.49.0
boto3==1.9.234
botocore==1.12.234
bz2file==0.98
cachetools==3.1.0
certifi==2019.9.11
cffi==1.13.0
chardet==3.0.4
Click==7.0
colorama==0.4.1
cooldict==1.4
cryptography==2.8
cycler==0.10.0
decorator==4.4.0
defusedxml==0.6.0
docutils==0.15.2
dpkt==1.9.2
entrypoints==0.3
filelock==3.0.12
funcsigs==1.0.2
future==0.17.1
gast==0.2.2
gensim==3.8.0
gitdb2==2.0.5
GitPython==2.1.11
google-auth==1.7.0
google-auth-oauthlib==0.4.1
grpcio==1.25.0
h5py==2.9.0
html5lib==1.0.1
humanfriendly==4.18
idalink==0.12
idna==2.8
imbalanced-learn==0.5.0
imblearn==0.0
importlib-metadata==0.23
ipykernel==5.1.2
ipython==7.8.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
itanium-demangler==1.0
jedi==0.15.1
Jinja2==2.10.3
jmespath==0.9.4
joblib==0.14.0
jsonschema==3.1.1
jupyter==1.0.0
jupyter-client==5.3.4
jupyter-console==6.0.0
jupyter-core==4.6.0
Keras==2.2.4
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.1
mistune==0.8.4
mkl-fft==1.0.14
mkl-random==1.1.0
mkl-service==2.3.0
mock==3.0.5
more-itertools==7.2.0
mulpyplexer==0.8
nbconvert==5.6.0
nbformat==4.4.0
networkx==2.3
notebook==6.0.1
numpy==1.17.2
oauthlib==3.1.0
olefile==0.46
opt-einsum==3.1.0
packaging==19.1
pandas==0.25.2
pandocfilters==1.4.2
parso==0.5.1
pbr==5.4.3
pefile==2019.4.18
pexpect==4.7.0
pickleshare==0.7.5
Pillow==6.2.0
pip==19.3.1
plac==1.1.0
plotly==4.1.1
pluggy==0.12.0
plumbum==1.6.7
prince==0.6.3
progressbar==2.5
prometheus-client==0.7.1
prompt-toolkit==2.0.10
protobuf==3.9.2
psutil==5.6.2
ptyprocess==0.6.0
py==1.8.0
pyasn1==0.4.7
pyasn1-modules==0.2.7
pyclustertend==1.3.2
pycparser==2.19
pydot==1.4.1
pyelftools==0.25
Pygments==2.4.2
pygraphviz==1.3
pyOpenSSL==19.0.0
pyparsing==2.4.2
pyrsistent==0.15.4
PySMT==0.8.0
PySocks==1.7.1
pytest==5.1.2
python-dateutil==2.8.0
pytz==2019.3
pyvex==8.19.7.25
PyYAML==5.1.2
pyzmq==18.1.0
qtconsole==4.5.5
ray==0.7.4
redis==3.3.8
requests==2.22.0
requests-oauthlib==1.3.0
retrying==1.3.3
rpyc==4.0.2
rsa==4.0
s3transfer==0.2.1
scikit-learn==0.21.3
scipy==1.3.1
Send2Trash==1.5.0
setuptools==41.4.0
six==1.12.0
smart-open==1.8.4
smmap2==2.0.5
sortedcontainers==2.1.0
tensorboard==2.0.1
tensorflow-estimator==2.0.1
tensorflow-gpu==2.0.0
termcolor==1.1.0
terminado==0.8.2
testpath==0.4.2
tornado==6.0.3
tqdm==4.36.1
traitlets==4.3.3
unicorn==1.0.1
urllib3==1.24.2
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.16.0
wheel==0.33.6
widgetsnbextension==3.5.1
wrapt==1.11.2
z3-solver==4.5.1.0.post2
zipp==0.6.0

Next steps

Issue description

I am unable to write a PR curve to TensorBoard with TF 2.0 I’ve attempted using the built-in summary builders and attempted to build my own summaries. None of them seem to work.

Are these intended to work in TF 2.0?

I will update this with whatever code is requested.

[nfelt]

We don’t yet have a TF 2.0-native version of the PR curve summary op, but it should be possible using an experimental write_raw_pb option to take output generated by the existing TF 1.x-era summary ops and feed them into the new TF 2.0 writing APIs, if that’s the roadblock you’re running into. For example:

import tensorflow as tf
import tensorboard.summary.v1 as tb_summary

pr_curve_summary = tb_summary.pr_curve(
    "pr", labels=tf.constant([True]), predictions=tf.constant([0.5]))

writer = tf.summary.create_file_writer("/tmp/pr_curves")
with writer.as_default():
  tf.summary.experimental.write_raw_pb(pr_curve_summary, step=1)

If that’s not the problem you’re running into, could you provide some short sample code that illustrates what you’re trying to do?

[ernesto-elsaesser]

Hi @nfelt, the workaround you posted above does work for me as well. I just wanted to ask whether you are actually still planning to migrate the PR Curve summary code to TF 2.0. In case this is very low prio or not currently planned, would you accept a PR?