Two problem about update function

I have two problem about update function after read your code. And could anyone explanation it for me? I am very appreciated. Firstly, I can’t understand what role does variable “num_sample” play when train q network?

# train q network
num_sample = 1
target_q = 0.0
for i in range(num_sample):
      target_act_next_n = [agents[i].p_debug['target_act'](obs_next_n[i]) for i in range(self.n)]
      target_q_next = self.q_debug['target_q_values'](*(obs_next_n + target_act_next_n))
      target_q += rew + self.args.gamma * (1.0 - done) * target_q_next
target_q /= num_sample

Secondly, why the loss of p should be loss = pg_loss + p_reg * 1e-3, and what role does p_reg play in the loss.

[Ah31]

p_reg is regularizing p_values obtained from the following:
p = p_func(p_input, int(act_pdtype_n[p_index].param_shape()[0]), scope="p_func", num_units=num_units). You can notice that p_values are being used to compute the pg_loss.

I don’t know yet why they have used num_sample because equalling it to 1 does not seem to do anything useful.

[hexbo]

p_reg is regularizing p_values obtained from the following: p = p_func(p_input, int(act_pdtype_n[p_index].param_shape()[0]), scope="p_func", num_units=num_units). You can notice that p_values are being used to compute the pg_loss.

I don’t know yet why they have used num_sample because equalling it to 1 does not seem to do anything useful.

There is two variables both named i.