RuntimeError: index -1 is out of bounds for dimension 1 with size 779

Hi, Thanks for sharing the implementation of TabNet, it is very useful. how to fix this. why I get this error?

the shape of df’s are: train_X.shape, Val_X.shape, train_y.shape, Val_y.shape ((74676, 779), (18670, 779), (74676,), (18670,))

Parameter using: clf = TabNetClassifier( # default tabnet hyperparameters n_d=64, n_a=64, n_steps=5, gamma=1.5, n_independent=2, n_shared=2, lambda_sparse=1e-4, momentum=0.3, clip_value=2., optimizer_fn=torch.optim.Adam, optimizer_params=dict(lr=2e-2), scheduler_params = {“gamma”: 0.95, “step_size”: 20}, scheduler_fn=torch.optim.lr_scheduler.StepLR, epsilon=1e-15 )

clf.fit( X_train=X_train, y_train=y_train, X_valid=X_valid, y_valid=y_valid, max_epochs=max_epochs, patience=100, batch_size=10000, virtual_batch_size=256 )

`Will train until validation stopping metric hasn’t improved in 50 rounds.

| EPOCH | train | valid | total time (s)

RuntimeError Traceback (most recent call last) <ipython-input-168-7ff3d7a869b5> in <module> 5 X_valid=X_valid, y_valid=y_valid, 6 max_epochs=max_epochs, patience=50, ----> 7 batch_size=10000, virtual_batch_size=256 8 )

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_model.py in fit(self, X_train, y_train, X_valid, y_valid, loss_fn, weights, max_epochs, patience, batch_size, virtual_batch_size, num_workers, drop_last) 226 self.learning_rates.append(self.optimizer.param_groups[-1][“lr”]) 227 –> 228 fit_metrics = self.fit_epoch(train_dataloader, valid_dataloader) 229 230 # leaving it here, may be used for callbacks later

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_model.py in fit_epoch(self, train_dataloader, valid_dataloader) 345 DataLoader with valid set 346 “”" –> 347 train_metrics = self.train_epoch(train_dataloader) 348 valid_metrics = self.predict_epoch(valid_dataloader) 349

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_model.py in train_epoch(self, train_loader) 609 610 for data, targets in train_loader: –> 611 batch_outs = self.train_batch(data, targets) 612 if self.output_dim == 2: 613 y_preds.append(torch.nn.Softmax(dim=1)(batch_outs[“y_preds”])[:, 1]

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_model.py in train_batch(self, data, targets) 652 self.optimizer.zero_grad() 653 –> 654 output, M_loss = self.network(data) 655 656 loss = self.loss_fn(output, targets)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_network.py in forward(self, x) 262 def forward(self, x): 263 x = self.embedder(x) –> 264 return self.tabnet(x) 265 266 def forward_masks(self, x):

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_network.py in forward(self, x) 134 135 for step in range(self.n_steps): –> 136 M = self.att_transformers[step](prior, att) 137 M_loss += torch.mean(torch.sum(torch.mul(M, torch.log(M+self.epsilon)), 138 dim=1))

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/tab_network.py in forward(self, priors, processed_feat) 308 x = self.bn(x) 309 x = torch.mul(x, priors) –> 310 x = self.selector(x) 311 return x 312

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/sparsemax.py in forward(self, input) 89 90 def forward(self, input): —> 91 return sparsemax(input, self.dim) 92 93

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/sparsemax.py in forward(ctx, input, dim) 41 max_val, _ = input.max(dim=dim, keepdim=True) 42 input -= max_val # same numerical stability trick as for softmax —> 43 tau, supp_size = SparsemaxFunction._threshold_and_support(input, dim=dim) 44 output = torch.clamp(input - tau, min=0) 45 ctx.save_for_backward(supp_size, output)

~/anaconda3/lib/python3.7/site-packages/pytorch_tabnet/sparsemax.py in _threshold_and_support(input, dim) 74 75 support_size = support.sum(dim=dim).unsqueeze(dim) —> 76 tau = input_cumsum.gather(dim, support_size - 1) 77 tau /= support_size.to(input.dtype) 78 return tau, support_size

RuntimeError: index -1 is out of bounds for dimension 1 with size 779`

[Optimox]

Glad to know that it’s working!

[Optimox]

@monk1337 actually this might be due to the last batch having only one element, could you set drop_last to True and see if it works for any batch_size?