Integrate with TensorFlow 2.0

In this PR I’ve started getting the tf 2.0 transition ready. Currently the tests are passing in the PR but its still using the 1.X API. In conftest.py its disabling the v2 behavior with tf.compat.v1.disable_v2_behavior(). As tests are moved over to the tf 2.0 we enable v2 behavior for those individual tests.

Doing this conversion and this doc from Tensorflow has given a good feel for what the major changes are going to be.

One interesting decision we need to make is whether we want to continue to be master compatible with the tf 2.0 changes (meaning master will continue to work with both tf 1.14 and tf 2.0 pip packages) or if we want to have a longer running branch where we put all the 2.0 changes into. I’m pretty open to either direction but think keeping tf 1.14 working will become more complicated as we convert more and more code to use tf 2.0 API.

Major Changes

I believe these can be completed independently for the most part.

In no particular order:

• Convert tests to use tf.function instead of tf.Session
• Figure out best way to configure devices etc like we do with tfe.Session without subclassing tf.Session especially for RemoteConfig.
• Figure out best way to enable eager execution. Thinking it might work to create something like a EagerConfig in the spirit LocalConfig and RemoteConfig
• Start converting any leftover uses of tf.compat.v1.*.
• Update any tfe API to match the corresponding v2 API in Tensorflow
• Update the converter to work with v2 API
• During the above conversion I noticed some oddities with the space_to_batch/batch_to_space API. Figure out this.
• Variables work quite a bit different with tf.function figure out how we can make this work

[justin1121]

One thing I don’t like is that we’re leaking implementation details by doing it that way. I suggest introducing a tfe.encrypted_computation decorator instead, along the lines of what TF Federated is doing. For now this decorator might simply forward everything to tf.function.

I’ll look into this a bit. Wasn’t sure exactly how this would work at first but now I have some ideas.

I’ve just started digging into the examples a bit which are more likely to be more nuanced around how they work compared to how some of the tests work which should give us some more idea around what we need to fix (e.g. what needs to be done for tfe.local_computation).

[skeydan]

Thanks @jvmncs for the update, and very happy to hear you liked the post! 😃