Speed up of Cfmessage

Hi there,

I have a need to process NOAA files for HRRR data and corresponding forecasts but I’m running into issues around speed of reading downloaded files for parsing. It is taking roughly 40 seconds to load data for one location alone. I’ve already managed to make gains with a few other tricks external to the library but when I now run a profiler on the code, about 20% of the overall total execution time of my task is spent inside functions belonging to just CfMessage alone and it is the biggest source of execution delay that isn’t attributed to xarray itself.

I’m very new to the internals of the code here, but at a glance a few areas of potential performance gains may involve:

• Replacing string keys with assigned integer based constants
• The addition of optional de-selection at a higher level of what content is needed from COMPUTED_KEYS as options like verifying_time or indexing_time do not have end use utility on my end.
• Dropping the LOG initialization inside CfMessage as it isn’t called elsewhere in file but still has an initialization overhead
• The if-else stack inside build_valid_time function could potentially be replaced with a more divide and conquer approach but this would be a very minor improvement

[alexamici]

@Snowda FYI we looked into the matter and in our tests you can gain around 15% of performance by not computing the time coordinate that you are not going to use. @aurghs will be working on a PR.

Everything else doesn’t appear to be worth the effort in our benchmarks.

[alexamici]

@aurghs: “Replacing string keys with assigned integer based constants” this looks like a very promising performance boost once we enable selecting the data type for keys (that we need to do anyway due to #195) and then fetching the string representation only once like we do with the non-dimensional keys. You may have a look at it once you have time.