Generating fonts from Designspace failed: unsigned short is greater than maximum [error when generating variable font]

INFO:fontTools.varLib:Merging TT hinting
DEBUG:fontTools.ttLib.ttFont:Compiling 'glyf' table
DEBUG:fontTools.ttLib.ttFont:Writing 'glyf' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'maxp' table
DEBUG:fontTools.ttLib.ttFont:Writing 'maxp' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'loca' table
DEBUG:fontTools.ttLib.ttFont:Writing 'loca' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'head' table
DEBUG:fontTools.ttLib.ttFont:Writing 'head' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'hmtx' table
DEBUG:fontTools.ttLib.ttFont:Writing 'hmtx' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'hhea' table
DEBUG:fontTools.ttLib.ttFont:Writing 'hhea' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'OS/2' table
DEBUG:fontTools.ttLib.ttFont:Writing 'OS/2' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'cmap' table
DEBUG:fontTools.ttLib.ttFont:Writing 'cmap' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'name' table
DEBUG:fontTools.ttLib.ttFont:Writing 'name' table to disk
DEBUG:fontTools.ttLib.ttFont:Compiling 'post' table
ERROR:root:In 'SHSans-VF-ufo-test.designspace': Generating fonts from Designspace failed: unsigned short is greater than maximum
Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontmake\font_project.py", line 1003, in run_from_designspace
    self._run_from_designspace_interpolatable(
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontmake\font_project.py", line 1086, in _run_from_designspace_interpolatable
    self.build_variable_font(
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontmake\font_project.py", line 342, in build_variable_font
    font = ufo2ft.compileVariableTTF(
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\ufo2ft\__init__.py", line 570, in compileVariableTTF
    return call_postprocessor(varfont, baseUfo, glyphSet=None, **kwargs)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\ufo2ft\__init__.py", line 74, in call_postprocessor
    postProcessor = postProcessorClass(otf, ufo, glyphSet=glyphSet)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\ufo2ft\postProcessor.py", line 50, in __init__
    self.otf = _reloadFont(otf)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\ufo2ft\postProcessor.py", line 394, in _reloadFont
    font.save(stream)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontTools\ttLib\ttFont.py", line 184, in save
    writer_reordersTables = self._save(tmp)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontTools\ttLib\ttFont.py", line 224, in _save
    self._writeTable(tag, writer, done, tableCache)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontTools\ttLib\ttFont.py", line 647, in _writeTable
    tabledata = self.getTableData(tag)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontTools\ttLib\ttFont.py", line 669, in getTableData
    return self.tables[tag].compile(self)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontTools\ttLib\tables\_p_o_s_t.py", line 51, in compile
    data = data + self.encode_format_2_0(ttFont)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontTools\ttLib\tables\_p_o_s_t.py", line 174, in encode_format_2_0
    indices.append(index)
OverflowError: unsigned short is greater than maximum

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontmake\__main__.py", line 602, in main
    project.run_from_designspace(inputs.designspace_path, **args)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\fontmake\font_project.py", line 1017, in run_from_designspace
    raise FontmakeError(
fontmake.errors.FontmakeError: In 'SHSans-VF-ufo-test.designspace': Generating fonts from Designspace failed: unsigned short is greater than maximum

How should I begin with debugging the issue? The above comes from generating fonts from two UFOs and a DesignSpace file.

[NightFurySL2001]

Adding “com.github.googlei18n.ufo2ft.keepGlyphNames” to False in lib.plist successfully exported the TTF variable font through compileVariableTTF. Some minor errors popped up (e.g. Cannot look up family name, assign the 'familyname' attribute to the default source., hb.repack failed to serialize 'GSUB', attempting fonttools resolutions ; the error message was: RepackerError) and some vmtx seems broken, but all looks good and fine so far.

I do agree to add using post table version 3 as a fontmake option is a good solution.

[anthrotype]

can you try to set this lib key “com.github.googlei18n.ufo2ft.keepGlyphNames” to False in all the master UFOs before calling compileVariableTTF? That will make the ufo2ft.postProcessor set the post to format 3.0 on the built VF which is what you want:

https://github.com/googlefonts/ufo2ft/blob/71ef6caa1f1fc63b487e9d92bd5735b5d117f506/Lib/ufo2ft/postProcessor.py#L67-L74

We should document this better, and maybe expose that as a fontmake option.