question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

votable parsing issue resulting `object` dtyped Table columns

See original GitHub issue
from astroquery.alma import Alma
payload = {'target_name':'M100', 'dataproduct_type':'cube'}
payload_query = Alma.query(payload=payload, public=True, science=True)
payload_query.write('m100.fits')

Throws ValueError: Illegal format object.

Possibly related to astropy/astropy#7480.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
github-actions[bot]commented, Feb 25, 2022

Welcome to Astropy 👋 and thank you for your first issue!

A project member will respond to you as soon as possible; in the meantime, please double-check the guidelines for submitting issues and make sure you’ve provided the requested details.

GitHub issues in the Astropy repository are used to track bug reports and feature requests; If your issue poses a question about how to use Astropy, please instead raise your question in the Astropy Discourse user forum and close this issue.

If you feel that this issue has not been responded to in a timely manner, please leave a comment mentioning our software support engineer @embray, or send a message directly to the development mailing list. If the issue is urgent or sensitive in nature (e.g., a security vulnerability) please send an e-mail directly to the private e-mail feedback@astropy.org.

0reactions
bsipoczcommented, Feb 25, 2022

This is not an astroquery issue, but very likely a votable parsing issue, which belongs to the core package. I would expect this to be basically a duplicate of https://github.com/astropy/astropy/issues/7480.

Here the difference is that there is an added layer of pyvo in the middle which does the communication with the TAP service. I would say this is a pyVO issue, it’s responsible for parsing the votable return of the server into an astropy Table.

here is some debug info from L242 of pyvo/pyvo/dal/query.py calling from astropy.io.votable import parse as votableparse that may help to identify the underlying parsing issue, this time with datatype=char and either "adql:TIMESTAMP" or "adql:REGION" as xtype

ipdb> aaa=votableparse(self.execute_stream(post=post).read)
ipdb> for i in aaa.iter_fields_and_params():
    i
 
 
 
<FIELD ID="obs_publisher_did" arraysize="33" datatype="char" name="obs_publisher_did" ucd="meta.ref.url;meta.curation" utype="obscore:Curation.PublisherDID"/>
<FIELD ID="obs_collection" arraysize="4" datatype="char" name="obs_collection" ucd="meta.id" utype="obscore:DataID.Collection"/>
<FIELD ID="facility_name" arraysize="3" datatype="char" name="facility_name" ucd="meta.id;instr.tel" utype="obscore:Provenance.ObsConfig.Facility.name"/>
<FIELD ID="instrument_name" arraysize="4" datatype="char" name="instrument_name" ucd="meta.id;instr" utype="obscore:Provenance.ObsConfig.Instrument.name"/>
<FIELD ID="obs_id" arraysize="64" datatype="char" name="obs_id" ucd="meta.id" utype="obscore:DataID.observationID"/>
<FIELD ID="dataproduct_type" arraysize="5" datatype="char" name="dataproduct_type" ucd="meta.id" utype="obscore:ObsDataset.dataProductType"/>
<FIELD ID="calib_level" datatype="int" name="calib_level" ucd="meta.code;obs.calib" utype="obscore:ObsDataset.calibLevel"/>
<FIELD ID="target_name" arraysize="256" datatype="char" name="target_name" ucd="meta.id;src" utype="obscore:Target.Name"/>
<FIELD ID="s_ra" datatype="double" name="s_ra" ucd="pos.eq.ra" unit="deg" utype="obscore:Char.SpatialAxis.Coverage.Location.Coord.Position2D.Value2.C1" xtype="adql:DOUBLE"/>
<FIELD ID="s_dec" datatype="double" name="s_dec" ucd="pos.eq.dec" unit="deg" utype="obscore:Char.SpatialAxis.Coverage.Location.Coord.Position2D.Value2.C2" xtype="adql:DOUBLE"/>
<FIELD ID="s_fov" datatype="double" name="s_fov" ucd="phys.angSize;instr.fov" unit="deg" utype="obscore:Char.SpatialAxis.Coverage.Bounds.Extent.diameter" xtype="adql:DOUBLE"/>
<FIELD ID="s_region" arraysize="*" datatype="char" name="s_region" ucd="phys.outline;obs.field" unit="deg" utype="obscore:Char.SpatialAxis.Coverage.Support.Area" xtype="adql:REGION"/>
<FIELD ID="s_resolution" datatype="double" name="s_resolution" ucd="pos.angResolution" unit="deg" utype="obscore:Char.SpatialAxis.Resolution.refval.value" xtype="adql:DOUBLE"/>
<FIELD ID="t_min" datatype="double" name="t_min" ucd="time.start;obs.exposure" unit="d" utype="obscore:Char.TimeAxis.Coverage.Bounds.Limits.StartTime" xtype="adql:DOUBLE"/>
<FIELD ID="t_max" datatype="double" name="t_max" ucd="time.end;obs.exposure" unit="d" utype="obscore:Char.TimeAxis.Coverage.Bounds.Limits.StopTime" xtype="adql:DOUBLE"/>
<FIELD ID="t_exptime" datatype="double" name="t_exptime" ucd="time.duration;obs.exposure" unit="s" utype="obscore:Char.TimeAxis.Coverage.Support.Extent" xtype="adql:DOUBLE"/>
<FIELD ID="t_resolution" datatype="double" name="t_resolution" ucd="time.resolution" unit="s" utype="obscore:Char.TimeAxis.Resolution.refval.value" xtype="adql:DOUBLE"/>
<FIELD ID="em_min" datatype="double" name="em_min" ucd="em.wl;stat.min" unit="m" utype="obscore:Char.SpectralAxis.Coverage.Bounds.Limits.LoLimit" xtype="adql:DOUBLE"/>
<FIELD ID="em_max" datatype="double" name="em_max" ucd="em.wl;stat.max" unit="m" utype="obscore:Char.SpectralAxis.Coverage.Bounds.Limits.HiLimit" xtype="adql:DOUBLE"/>
<FIELD ID="em_res_power" datatype="double" name="em_res_power" ucd="spect.resolution" utype="obscore:Char.SpectralAxis.Resolution.ResolPower.refval" xtype="adql:DOUBLE"/>
<FIELD ID="pol_states" arraysize="64" datatype="char" name="pol_states" ucd="meta.code;phys.polarization" utype="obscore:Char.PolarizationAxis.stateList"/>
<FIELD ID="o_ucd" arraysize="35" datatype="char" name="o_ucd" ucd="meta.ucd" utype="obscore:Char.ObservableAxis.ucd"/>
<FIELD ID="access_url" arraysize="72" datatype="char" name="access_url" ucd="meta.ref.url" utype="obscore:Access.Reference"/>
<FIELD ID="access_format" arraysize="9" datatype="char" name="access_format" ucd="meta.code.mime" utype="obscore:Access.Format"/>
<FIELD ID="proposal_id" arraysize="64" datatype="char" name="proposal_id" xtype="adql:VARCHAR"/>
<FIELD ID="data_rights" arraysize="11" datatype="char" name="data_rights" xtype="adql:VARCHAR"/>
<FIELD ID="gal_longitude" datatype="double" name="gal_longitude" ucd="pos.galactic.lon" unit="deg" xtype="adql:DOUBLE"/>
<FIELD ID="gal_latitude" datatype="double" name="gal_latitude" ucd="pos.galactic.lat" unit="deg" xtype="adql:DOUBLE"/>
<FIELD ID="band_list" arraysize="30" datatype="char" name="band_list" xtype="adql:VARCHAR"/>
<FIELD ID="em_resolution" datatype="double" name="em_resolution" ucd="spect.resolution" unit="m" utype="obscore:Char.SpectralAxis.Resolution.refval" xtype="adql:DOUBLE"/>
<FIELD ID="bandwidth" datatype="double" name="bandwidth" ucd="em.freq;instr.bandpass" unit="Hz" utype="obscore:Char.SpectralAxis.Coverage.Bounds.Extent" xtype="adql:DOUBLE"/>
<FIELD ID="antenna_arrays" arraysize="660" datatype="char" name="antenna_arrays" ucd="meta.code.member;instr.setup" xtype="adql:VARCHAR"/>
<FIELD ID="is_mosaic" arraysize="1" datatype="char" name="is_mosaic" xtype="adql:CHAR"/>
<FIELD ID="obs_release_date" arraysize="*" datatype="char" name="obs_release_date" ucd="time.release_date" utype="obscore:Curation.releaseDate" xtype="adql:TIMESTAMP"/>
<FIELD ID="spatial_resolution" datatype="double" name="spatial_resolution" unit="arcsec" xtype="adql:DOUBLE"/>
<FIELD ID="frequency_support" arraysize="4000" datatype="char" name="frequency_support" ucd="em.freq;obs;meta.main" unit="GHz" utype="obscore:Char.SpectralAxis.Coverage.Location.support" xtype="adql:VARCHAR"/>
<FIELD ID="frequency" datatype="double" name="frequency" ucd="em.freq;obs;meta.main" unit="GHz" utype="obscore:Char.SpectralAxis.Coverage.Location.refval" xtype="adql:DOUBLE"/>
<FIELD ID="velocity_resolution" datatype="double" name="velocity_resolution" ucd="phys.veloc;spect.resolution" unit="m / s" utype="obscore:Char.VelocityAxis.Resolution.refval" xtype="adql:DOUBLE"/>
<FIELD ID="obs_creator_name" arraysize="256" datatype="char" name="obs_creator_name" ucd="meta.id.PI" xtype="adql:VARCHAR"/>
<FIELD ID="pub_title" arraysize="256" datatype="char" name="pub_title" xtype="adql:VARCHAR"/>
<FIELD ID="first_author" arraysize="256" datatype="char" name="first_author" xtype="adql:VARCHAR"/>
<FIELD ID="authors" arraysize="4000" datatype="char" name="authors" xtype="adql:VARCHAR"/>
<FIELD ID="pub_abstract" arraysize="4000" datatype="char" name="pub_abstract" xtype="adql:VARCHAR"/>
<FIELD ID="publication_year" datatype="int" name="publication_year"/>
<FIELD ID="proposal_abstract" arraysize="4000" datatype="char" name="proposal_abstract" ucd="meta.abstract" xtype="adql:VARCHAR"/>
<FIELD ID="schedblock_name" arraysize="128" datatype="char" name="schedblock_name" ucd="meta.code;obs;instr.setup" xtype="adql:VARCHAR"/>
<FIELD ID="proposal_authors" arraysize="2000" datatype="char" name="proposal_authors" ucd="meta.id.CoI" xtype="adql:VARCHAR"/>
<FIELD ID="sensitivity_10kms" datatype="double" name="sensitivity_10kms" unit="mJy / beam" xtype="adql:DOUBLE"/>
<FIELD ID="cont_sensitivity_bandwidth" datatype="double" name="cont_sensitivity_bandwidth" unit="mJy / beam" xtype="adql:DOUBLE"/>
<FIELD ID="pwv" datatype="float" name="pwv" ucd="obs.atmos.extinction" unit="mm" xtype="adql:FLOAT"/>
<FIELD ID="group_ous_uid" arraysize="64" datatype="char" name="group_ous_uid" xtype="adql:VARCHAR"/>
<FIELD ID="member_ous_uid" arraysize="64" datatype="char" name="member_ous_uid" xtype="adql:VARCHAR"/>
<FIELD ID="asdm_uid" arraysize="32" datatype="char" name="asdm_uid" ucd="meta.id" xtype="adql:VARCHAR"/>
<FIELD ID="obs_title" arraysize="256" datatype="char" name="obs_title" ucd="meta.title;obs.proposal" xtype="adql:VARCHAR"/>
<FIELD ID="type" arraysize="16" datatype="char" name="type" xtype="adql:VARCHAR"/>
<FIELD ID="scan_intent" arraysize="256" datatype="char" name="scan_intent" ucd="meta.code.class;obs" xtype="adql:VARCHAR"/>
<FIELD ID="science_observation" arraysize="1" datatype="char" name="science_observation" ucd="meta.code.class;obs" xtype="adql:VARCHAR"/>
<FIELD ID="spatial_scale_max" datatype="double" name="spatial_scale_max" ucd="pos.angResolution;stat.max" unit="arcsec" utype="obscore:Characterisation.SpatialAxis.Coverage.Bounds.limits.Interval.HiLim" xtype="adql:DOUBLE"/>
<FIELD ID="qa2_passed" arraysize="1" datatype="char" name="qa2_passed" ucd="meta.flag" xtype="adql:CHAR"/>
<FIELD ID="bib_reference" arraysize="4000" datatype="char" name="bib_reference" xtype="adql:VARCHAR"/>
<FIELD ID="science_keyword" arraysize="200" datatype="char" name="science_keyword" xtype="adql:VARCHAR"/>
<FIELD ID="scientific_category" arraysize="200" datatype="char" name="scientific_category" xtype="adql:VARCHAR"/>
<FIELD ID="lastModified" arraysize="*" datatype="char" name="lastModified" xtype="adql:TIMESTAMP"/>
ipdb> 
Read more comments on GitHub >

github_iconTop Results From Across the Web

votable parsing issue: datatype="char" casted as object dtype ...
I downloaded a table from the Gaia science archive using astroquery, and tried to write it to a FITS file: from astroquery.gaia import...
Read more >
VOTable XML Handling (astropy.io.votable) — Astropy v5.2
Table object to an astropy.table.Table object, you can specify whether to give preference to name or ID attributes when naming the columns.
Read more >
vo.table API documentation - stsdas
Parses a VOTABLE xml file (or file-like object), and returns a VOTable object, ... it is used for the column name in the...
Read more >
Pyvo TAP result to csv (via astropy.Table?)
I have a TAP table in a VO service that I want to dump to csv. ... suspect this could rather be a...
Read more >
STIL - Starlink Tables Infrastructure Library
7.4.1 Generic table output; 7.4.2 Single VOTable output ... Every object in the column described by that metadata should be an instance of ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found