question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

UnicodeDecodeError Saving votables from astroquery searches

See original GitHub issue

I am encountering UnicodeDecodeErrors while attempting to write astroquery results to votables. An example query which fails upon trying to save to votable format is:

from astroquery.alma import Alma
import astropy.units as u
res = Alma.query_object('PKS1830-211', 1 * u.arcmin)
res.write('PKS1830-211.xml', format='votable')

The resulting error is:

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-7-1afec7db28b5> in <module>()
----> 1 res.write('PKS1830-211.xml', format='votable')

/usr/lib/python3.6/site-packages/astropy/table/table.py in write(self, *args, **kwargs)
   2548         passed through to the underlying data reader (e.g. `~astropy.io.ascii.write`).
   2549         """
-> 2550         io_registry.write(self, *args, **kwargs)
   2551
   2552     def copy(self, copy_data=True):

/usr/lib/python3.6/site-packages/astropy/io/registry.py in write(data, *args, **kwargs)
    579
    580     writer = get_writer(format, data.__class__)
--> 581     writer(data, *args, **kwargs)
    582
    583

/usr/lib/python3.6/site-packages/astropy/io/votable/connect.py in write_table_votable(input, output, table_id, overwrite, tabledata_format)
    160
    161     # Write out file
--> 162     table_file.to_xml(output, tabledata_format=tabledata_format)
    163
    164

/usr/lib/python3.6/site-packages/astropy/io/votable/tree.py in to_xml(self, fd, compressed, tabledata_format, _debug_python_based_parser, _astropy_version)
   3484                 for element_set in element_sets:
   3485                     for element in element_set:
-> 3486                         element.to_xml(w, **kwargs)
   3487
   3488     def iter_tables(self):

/usr/lib/python3.6/site-packages/astropy/io/votable/tree.py in to_xml(self, w, **kwargs)
   3176                                 self.resources):
   3177                 for element in element_set:
-> 3178                     element.to_xml(w, **kwargs)
   3179
   3180     def iter_tables(self):

/usr/lib/python3.6/site-packages/astropy/io/votable/tree.py in to_xml(self, w, **kwargs)
   2753                 with w.tag('DATA'):
   2754                     if format == 'tabledata':
-> 2755                         self._write_tabledata(w, **kwargs)
   2756                     elif format == 'binary':
   2757                         self._write_binary(1, w, **kwargs)

/usr/lib/python3.6/site-packages/astropy/io/votable/tree.py in _write_tabledata(self, w, **kwargs)
   2778                 tablewriter.write_tabledata(
   2779                     w.write, array.data, array.mask, fields,
-> 2780                     supports_empty_values, indent, 1 << 8)
   2781             else:
   2782                 write = w.write

/usr/lib/python3.6/site-packages/astropy/io/votable/converters.py in output(self, value, mask)
    354             return ''
    355         if not isinstance(value, str):
--> 356             value = value.decode('ascii')
    357         return xml_escape_cdata(value)
    358

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 211: ordinal not in range(128)

A temporary workaround for me has been to exit votable/converters.py to all decode to utf-8 instead of ascii, but I am not sure this is a robust fix. I saw there were some other issues relating to UnicodeDecodeErrors, but not seemed to specifically address this. But if I’ve missed a known fix or workaround, please let me know.

My software versions are:

  • python: 3.6.3
  • astropy: 2.0.2
  • numpy: 1.13.3

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
privongcommented, Nov 14, 2017

@pllim I think the bug is inside astropy, not astoquery? astroquery.alma.Alma.query_object() returns an astropy Table object (https://astroquery.readthedocs.io/en/latest/api/astroquery.alma.AlmaClass.html#astroquery.alma.AlmaClass.query_object and http://docs.astropy.org/en/latest/api/astropy.table.Table.html#astropy.table.Table). So the write() function is in astropy.

1reaction
bsipoczcommented, Nov 13, 2017

@pllim - can you open an issue for that (or a fix)?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Source code for astroquery.esa.hubble.core
If the filename is defined, the file will be automatically saved output_format : string results format. Options are: 'votable': str, binary VOTable format ......
Read more >
Full Changelog — Astropy v5.2
Fixed a bug where astropy.io.votable.validate was printing output to sys.stdout when the ... Use astropy.samp for SAMP and astroquery for VO cone search....
Read more >
astroquery sometimes works fine with query result but most of ...
AppData\Local\Programs\Python\Python36\lib\site-packages\astroquery\utils\ ... votable {main_id,coordinates} votable open query coo 21:06:54 +38:44:58 ...
Read more >
Save FITS table: The keyword description with its value is too ...
I get an error when trying to save astropy Tables retrieved using astroquery to FITS files. In some case it complains that the...
Read more >
astroquery Documentation - Read the Docs
Queries that support a cone search with a specified radius ... The SIMBAD query tool creates a script query that returns VOtable XML...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found