Exporter performance

Several test models I’ve used lately, including the sample WaterBottle model, have a very small amount of mesh data, and yet still take a long time to export compared to other Blender exporters.

I’ve been investigating the speed of the exporter (mostly with some hacked logging of times).

Turns out, for the models I’ve been testing, the performance bottleneck is exporting the images. In particular, the __get_image_data function spends several seconds per exported texture, slowly dismantling the color channels and reassembling them. This is really only needed for the OcclusionRoughnessMetallic texture, and only when the user’s mapping of that texture’s channels do not match the glTF spec. For other channels like BaseColor, Normal, Emission, and correctly baked ORM textures (such as from imported models), this is wasted export time.

But even more time-consuming than that, is a little parameter here in the PNG writer:

https://github.com/KhronosGroup/glTF-Blender-IO/blob/4c15398154af66e817f66a53fab3d4f03662238d/addons/io_scene_gltf2/io/exp/gltf2_io_image_data.py#L135

The number 9 here is the compression level setting, being passed to zlib. From the zlib manual:

#define Z_NO_COMPRESSION         0
#define Z_BEST_SPEED             1
#define Z_BEST_COMPRESSION       9
#define Z_DEFAULT_COMPRESSION  (-1)

The compression level must be Z_DEFAULT_COMPRESSION, or between 0 and 9: 1 gives best speed, 9 gives best compression, 0 gives no compression at all (the input data is simply copied a block at a time). Z_DEFAULT_COMPRESSION requests a default compromise between speed and compression (currently equivalent to level 6).

I tried setting this to -1 for the default compression, and saw a significant speedup but some cost in file size. I hesitate to add yet another option to our already overwhelming list of export options, but this one has a big effect on time spent and resulting size.

And of course, if the exporter already has access to a source PNG or JPG, it should be just copying the already-compressed data from there directly, not re-running the zlib compression as that takes a lot of time.

[donmccurdy]

It would be helpful to know the feature(s) involved when performance is an issue – you can identify this by disabling animation, materials, etc. and comparing the export time. Currently we’ve identified bottlenecks in:

• Writing textures
• Writing animations

There may be others.

Textures were/are likely the main issue here, and @julienduroure has submitted a significant improvement there. And as @emackey mentions above, the exporter really should avoid re-compressing texture data unless it’s necessary.

@wcbx I understand your frustration with the issue, but please keep in mind that this is not affecting all users, and feedback of the form “This exporter was not stable enough for release” is neither constructive nor productive in finding a solution. You’ve commented in a couple places that you’re unhappy with performance, but haven’t (as far as I can tell) provided an example that we can debug.

[pjoe]

Embedded output (.glb or .gltf with embedded textures) currently always encodes in .png 😦

This for me ruins glb output as it takes my files from e.g. 800K to 8M. As mentioned by others: whenever possible output format should be preserved form the source image. And when we need conversion (for ORM textures) it would be really nice to have the option to specify output format - for web content size matters a lot, and factor 10 difference is a killer.