question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Control dtype with ascii.read (converters API needs better doc or refactoring)

See original GitHub issue

I cannot find a way to control the dtype of the output table when reading a file into a Table. Consider the following MWE, with 3 numerical columns, while one of them would preferably be kept as a string:

>>> from astropy.io import ascii

>>> indata = ("# This is a dummy file\n" 
...           "# with some text to ignore, and a header with column names\n" 
...           "# ra dec objid\n" 
...           "1 2 345\n" 
...           "3 4 456\n") 

>>> ascii.read(indata, format='commented_header', header_start=2, guess=False, fast_reader=False)
<Table length=2>
  ra   dec  objid
int64 int64 int64
----- ----- -----
    1     2   345
    3     4   456

>>> ascii.read(indata, format='commented_header', header_start=2, dtye=('i8', 'i8', 'S10'), guess=False, fast_reader=False)
TypeError: __init__() got an unexpected keyword argument 'dtye'

Reading in the same with np.loadtxt and then converting to a Table works, but it should ideally be supported directly.

import numpy as np
from astropy.table import Table

>>> Table(np.loadtxt('/tmp/a', dtype=[('ra', 'i8'), ('dec', 'i8'), ('objid', 'S10')]))
<Table length=2>
  ra   dec   objid 
int64 int64 bytes10
----- ----- -------
    1     2     345
    3     4     456

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:12 (12 by maintainers)

github_iconTop GitHub Comments

3reactions
taldcroftcommented, Jan 27, 2021

I’ve implemented this in a few lines of code. As always the pain is testing, docs etc. But maybe there will be a PR on the way.

In [2]: >>> ascii.read(indata, format='commented_header', header_start=2, dtype=('i8', 'i4', 'S10'), guess=False, fast_reader=False)
Out[2]: 
<Table length=2>
  ra   dec   objid 
int64 int32 bytes10
----- ----- -------
    1     2     345
    3     4     456
1reaction
bsipoczcommented, Jan 27, 2021

Oh, yes, indeed, this is exactly what I need. One minor comment though, it would be helpful to have the word dtype somewhere in the docs, as I was searching for dtype in that and many other docs pages without any useful results. (maybe it’s just me, that case this can be closed without a docs change, otherwise this can be a good “first issue”).

It’s also not clear what the “previous section” is referred to in These take advantage of the convert_numpy() function which returns a two-element tuple (converter_func, converter_type) as described in the previous section.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Reading Tables — Astropy v5.2
ascii converts the raw string values from the table into numeric data types by using converter functions such as the Python int and...
Read more >
What's New — pandas 0.15.0 documentation
This submodule now uses httplib2 and the Google apiclient and oauth2client API client libraries which should be more stable and, therefore, reliable than...
Read more >
Changes in rpy2 — rpy2 3.5.4 documentation
Pandas converter no longer fails with arrays of dtype pandas. ... rpy2.robjects.lib.ggplot2 maps more functions in the R package (issue #767).
Read more >
h5py Documentation - Read the Docs
dset.dtype ... One of the best features of HDF5 is that you can store metadata ... Building h5py also requires several Python packages, ......
Read more >
Changelog - Dask documentation
Documentation¶ · Better SEO for Installing Dask and Dask DataFrame Best Practices (GH#9178) Sarah Charlotte Johnson · Update logos page in docs (GH#9167)...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found