[MusicBERT]: How to fill masked tokens in an input sequence after training?

Hello again,

I have fine-tuned MusicBERT on masked language modeling using a custom dataset. I have loaded the fine-tuned checkpoint using:

roberta = RobertaModel.from_pretrained( # MusicBERTModel.from_pretrained also works
    '.',
    checkpoint_file=sys.argv[1],
    data_name_or_path=sys.argv[2],
    user_dir='musicbert'
)

What I want to do is to give it an input sequence, mask one or more tokens before passing the input to the model and somehow predict them. Something like masked language modeling, but with control over which tokens I want to mask and predict.

What I cannot understand is what format the input sequence should be in order to be passed to the model, and how to make the model predict the masked tokens in the input. I have tried to replicate it by looking at the fairseq’s training code since I want to do something similar, but it’s too complicated.

Thanks in advance.

[aspil]

Hello @mlzeng and @tripathiarpan20 I’ll be closing this issue as it is more than resolved. The MIDIformers project is indeed brilliant!

[mlzeng]

Hi @tripathiarpan20

The MIDI remixing tool works well. It’s a brilliant project! I have added the links in README.md.