interpolate_pos_embed on MultiTask

Hi, it looks like this parameter from the multitask is not being properly inherited by the input_pipeline.

from finetune import Classifier, MultiTask

MAX_LENGTH = 300
finetune_config = {'batch_size': 4, 
                  'interpolate_pos_embed': True,
                   'n_epochs' : 1, #default 3
                   'train_embeddings': False, 
                   'num_layers_trained': 3, 
                   'max_length': MAX_LENGTH
                  }
multi_model = MultiTask({"sentiment": Classifier, 
                         "tags": Classifier}, 
                        **finetune_config)

The previous code builds the multitask object.

The next code works fine finetuning it:

multi_model.finetune(X={"sentiment": X_train.regex_text.values,
                         "tags": X_train.regex_text.values}, 
                     Y={"sentiment": y_train.sentiment,
                         "tags": y_train.full_topic},
                     batch_size=4
                   )

Also, multi_model.input_pipeline.config['interpolate_pos_embed'] = True is verified.

But when prediction time comes:

y_pred = multi_model.predict({"sentiment": X_train.regex_text.values,
                                   "tags": X_train.regex_text.values})

It does not work with:

ValueError: Max Length cannot be greater than 300 if interpolate_pos_embed is turned off

I do not know if I am missing something on the setup or it is a conflict between the parameters of the distinct objects.

Thanks very much Madison for the great job! The MultiTask model is a fantastic tool for uneven multiobjective labeled data.

[benleetownsend]

@Guillermogsjc You’re correct in identifying that there is a problem here, in fact none of the model config is currently being inherited by predict. I will work on a fix for this now. You should be fine to run the above with default max_length=512 in the meanwhile.

[madisonmay]

Awesome, glad to hear it 😃

Going to go ahead and close this issue then!