activation function not work properly

import tensorflow as tf
import tf_encrypted as tfe
import numpy as np

val = 6

x = tf.constant(value=val, dtype=tf.float32)

def asdf():
    global x
    return x

def qwer():
    return tf.random_normal(shape=(1, 5), dtype=tf.float32)

def fdsa(x):
    return tf.print(x)

with tf.Session() as sess:
    z1 = sess.run(x)

tfe_x = tfe.define_private_input("a", asdf)
W = tfe.define_private_input("a", qwer)
r = tfe_x * W
relu_r = tfe.relu(r)
op = tfe.define_output("a", r, fdsa)
relu_op = tfe.define_output("a", relu_r, fdsa)

with tfe.Session() as sess:
    sess.run(tfe.global_variables_initializer())
    sess.run(op)
    sess.run(relu_op)

I write my code like this to check that relu works fine.

[[6.2624599908550529 -4.1710105166895293 -0.20941929583904892 10.096936442615455 8.4179240969364422]] [[-0.15759792714525225 4.9518366102728244 -0.032464563328760861 0.425849718030788 -2.7239750038103949]]

and I get answer like this.

answer for r is nearly what I expected but result for relu_r is not what I expected. I want to know what I miss from my code.

[jvmncs]

This is the same problem as with the approximate relu – for sigmoid there is no exact protocol, so we approximate it with polynomials. The polynomial approximation gets worse outside of a thin interval (something like -4 to 4), so the values start to explode outside of the normal range when you get outside of that interval.

If you switch val to something like 1 or -1 you should see better results.

[mortendahl]

Hi @ByungCheolShin, thanks a lot for giving it a try!

The problem is that by default we’re using an approximation to ReLU for better performance. Simply add tfe.set_protocol(tfe.protocol.SecureNN()) directly after your imports and you should be good.

Note that you are computing r and relu_r on two different values since every sess.run will re-run tf.random_normal; if this is not what you intended then simply use op = tfe.define_output("a", [r, relu_r], fdsa) instead of the two define_output you have now (changing the signature of fdsa to match).