What's the difference from DARTS?

Thanks for sharing the code.

I have a question about the implementation difference from DARTS. The training code looks like very similar to DARTS(https://github.com/quark0/darts).

As you mentioned in the paper, “2. Instead of using the whole DAG, GDAS samples one sub-graph at one training iteration, accelerating the searching procedure. Besides, the sampling in GDAS is learnable and contributes to finding a better cell.”

But in the forward function of MixedOp, the output is just the weighted sum of all ops, same as DARTS.

def forward(self, x, weights): return sum(w * op(x) for w, op in zip(weights, self._ops))

So, can you point out the code that “samples one sub-graph at one training iteration”? Thanks.

[D-X-Y]

Sorry for the confusion. That file is DARTS instead of our algorithm, we did not release the searching codes of GDAS. The main difference between GDAS and DARTS is that we use Gumbel-softmax with an acceleration trick to allow only one candidate CNN is used during forwarding, while can still back-propagate to the architecture parameters.

[buttercutter]

in the forward procedure, we only need to calculate the function Farg max(hi,j ) . During the backward procedure, we only back-propagate the gradient generated at the arg max(h̃i,j ).

@D-X-Y

In the quoted text above inside gdas paper, I have few questions :

1. I suppose argmax operation is not differentiable in pytorch ?
2. If not back-propagate all the other gradients, then the computational graph will be broken or detached in some way ?
3. Does this gumbel-softmax trick need to be applied for both training (for W training) and validation (for A training) datasets ?