The configuration of 3.0.2 and 3.1.0 is not compatible

The configuration of 3.0.2 and 3.1.0 is not compatible.

For example, in https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L388, config.chunk_size_feed_forward should change to config.get('chunk_size_feed_forward', 0), because there is not chunk_size_feed_forward config in former version.

class BertLayer(nn.Module):
    def __init__(self, config):
        super().__init__()
        # self.chunk_size_feed_forward = config.chunk_size_feed_forward
        self.chunk_size_feed_forward = config.get('chunk_size_feed_forward', 0)
        self.seq_len_dim = 1
        self.attention = BertAttention(config)
        self.is_decoder = config.is_decoder
        self.add_cross_attention = config.add_cross_attention
        if self.add_cross_attention:
            assert self.is_decoder, f"{self} should be used as a decoder model if cross attention is added"
            self.crossattention = BertAttention(config)
        self.intermediate = BertIntermediate(config)
        self.output = BertOutput(config)

[patrickvonplaten]

I see what you mean! From 3.1.0 onwards every configuration has a config.chunk_size_feed_forward parameter. So as far as I can see whenever a config is loaded (whether via .from_pretrained() or with BertConfig(....), this parameter is part of the config…). Can you give me an example where this would not be the case?

[franciszzj]

@patrickvonplaten In the future, I will make more normative to call modules. Thanks for your replay. I close this issue.