[BUG] ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 2215158447

@jcwchen Optimizing large models fails in the latest release of onnx (1.12.0) even with use_external_data_format=True.

In the latest version of onnxruntime, calling OnnxModel.save(self.model, output_path, use_external_data_format, all_tensors_to_one_file) fails with the following stack trace:

True Traceback (most recent call last): File “examples/onnxruntime/optimization/question-answering/run_qa.py”, line 525, in main() File “examples/onnxruntime/optimization/question-answering/run_qa.py”, line 311, in main optimizer.export( File “/home/mroyzen/train_files/optimum/optimum/onnxruntime/optimization.py”, line 150, in export optimizer.save_model_to_file(onnx_optimized_model_output_path, use_external_data_format=True) File “/opt/conda/lib/python3.8/site-packages/onnxruntime/transformers/models/gpt2/…/…/onnx_model.py”, line 938, in save_model_to_file OnnxModel.save(self.model, output_path, use_external_data_format, all_tensors_to_one_file) File “/opt/conda/lib/python3.8/site-packages/onnxruntime/transformers/models/gpt2/…/…/onnx_model.py”, line 914, in save save_model( File “/opt/conda/lib/python3.8/site-packages/onnx/init.py”, line 202, in save_model s = _serialize(proto) File “/opt/conda/lib/python3.8/site-packages/onnx/init.py”, line 71, in _serialize result = proto.SerializeToString() ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 2215158447

Linking my onnxruntime issue as well, and I believe this feature request is related. Your help would be appreciated. Thanks!

[JingyaHuang]

Hi @jcwchen,

Thanks for the suggestion!

I just found that actually the difference came from the fact that I was using optimization example instead of the quantization example. Now I have successfully exported the proto.

Thank you so much for helping, I will integrate these large proto export features in optimum. Thanks again for your help!

[jcwchen]

Thanks for the update and experiment. One more possible variance I can think of: did you run run_qa.py on optimum 1.12.3’s commit? (5a0106d781a8358aa7eab88a8b115a538b4840d1)