VOTable binary2 file reads back in with only one row

I have encountered a case where a VOTableFile that writes out correctly in tabledata format, apparently writes only one row in binary2 format. Furthermore, validate reports that the binary2 file is fine. The STILTS votlint checker from Starlink finds the 1-row problem.

To reproduce:

from astropy.utils.data import download_file
from astropy.io.votable import parse, validate

tabledata_file = download_file('http://web.ipac.caltech.edu/staff/shupe' +
                               '/firefly_testdata/tabledata-wide.xml',
                               cache=True)

print('validating downloaded file:')
validate(tabledata_file)
votablefile = parse(tabledata_file)
print('tabledata format: shape is ', votablefile.get_first_table().array.shape)

intermediate_name = 'tabledata-wide.xml'
votablefile.to_xml(intermediate_name, tabledata_format='tabledata')
print('validating the table when written as tabledata format:')
validate(intermediate_name)
int_votablefile = parse(intermediate_name)
print('tabledata format: shape is ', int_votablefile.get_first_table().array.shape)

outname = 'binary2-wide.xml'
int_votablefile.to_xml(outname, tabledata_format='binary2')

print('validating binary2 output:')
validate(outname)

vobinary2 = parse(outname)
print('binary2 format: shape is ', vobinary2.get_first_table().array.shape)

STILTS votlint passes the tabledata versions. For the binary2 file it reports:

ERROR (l.4047, c.314415): Stream ended during data read; done 2/4
ERROR (l.4047, c.314415): End of stream while reading 1134821376 elements (probable stream corruption)
ERROR (l.4047, c.314415): Streaming error java.io.EOFException
ERROR (l.4055, c.11): Row count (1) not equal to nrows attribute (49)

EDIT: Syntax highlighting

[pllim]

@stargaser , okay. I’ll tack on your metadata problem onto #7759. Thank you!

[pllim]

Indeed metadata is not well supported in the unified I/O interface. I am not sure if I have time to look into this. But I am of the opinion that if we were to spend time fixing this issue up, the effort is better spent on implementing metadata support than figuring out why votable.parse etc did not behave the same as unified I/O. If LSST would like to provide resource to implement metadata support, I would be happy to help with the onboarding process!