question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Quantity objects should accept pandas Series

See original GitHub issue

Where Astropy expects a Quantity object, it rejects as input a pandas.Series * astropy unit. I believe this is unnecessarily strict, since a numpy array * astropy unit would be permitted. Here is an example of astropy rejecting what I believe should be a valid input of pandas.Series() * u.unit:

----> 1 s1723_spec = specutils.Spectrum1D(spectral_axis = df_s1723['wave'] * u.AA, flux=df_s1723['fnu'] * u.Unit('Jansky'))

~/anaconda3/envs/astroconda/lib/python3.6/site-packages/specutils/spectra/spectrum1d.py in init(self, flux, spectral_axis, wcs, velocity_convention, rest_value, redshift, radial_velocity, bin_specification, **kwargs)
97 if flux is not None:
98 if not isinstance(flux, u.Quantity):
---> 99 raise ValueError("Flux must be a Quantity object.")
100 elif flux.isscalar:
101 flux = u.Quantity([flux])

ValueError: Flux must be a Quantity object.

The kludgy workaround is to put .values after each Series: s1723_spec = specutils.Spectrum1D(spectral_axis = df_s1723['wave'].values * u.AA, flux=df_s1723['fnu'].values * u.Unit('Jansky'))

However, the workaround is kludgy, and I think unfair to pandas users. A Pandas.Series() will act like a numpy array when it needs to. Here, Astropy is rejecting it before it can.

Please see https://github.com/astropy/specutils/issues/740 , because I discovered the issue in the specutils.Spectrum1D class, although I believe the problem actually lies with Quantities.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

3reactions
mhvkcommented, Jan 13, 2021

I’m actually a bit surprised it does not work, as inside Quantity it just tries to convert any otherwise unrecognized input to ndarray and then turn it into a Quantity. So, perhaps good to see what actually happens:

import pandas as pd, numpy as np, astropy.units as u
a = pd.Series([1, 2., 3.])
np.array(a)  # works
# array([1., 2., 3.])
u.Quantity(a, u.m)  # should work too. It does!
# <Quantity [1., 2., 3.] m>
b = a * u.m
b  # Oops
# UnitConversionError: Can only apply 'greater' function to dimensionless quantities when other argument is not a quantity (unless the latter is all zero/infinity/nan)
type(b)  # Oops
# pandas.core.series.Series
b.values  # Interesting
<Quantity [1., 2., 3.] m>
u.m.__rmul__(a)  # works
# <Quantity [1., 2., 3.] m>

So, the problem seems to be that for multiplication, pd.Series thinks it can handle units, even though obviously it cannot. In that sense, arguably the problem is with pandas rather than astropy; it should really return NotImplemented (or rely less on arbitrary multiplication to give something that it still understands – note that this really is not a trivial problem, I can definitely see the reasoning behind the approach they took…).

Anyway, bottom line is that the exact request is not something we can solve in astropy: pandas just takes care of the multiplcation and we get no chance to take control. But there is a good work around for converting a Series to a Quantity: one can just use the class initializer (which will work for everything else too, of course).

Interestingly, what works as well (somewhat coincidentally, just because it happens not to be defined by pandas) is the in-place shift operator:

a << u.m
# <Quantity [1., 2., 3.] m>
0reactions
apurvapatkarcommented, Mar 16, 2021

Thanks for the reply. I would be happy to assist in documenting this issue and the approach.

Read more comments on GitHub >

github_iconTop Results From Across the Web

pandas.Series — pandas 1.5.2 documentation
Labels need not be unique but must be a hashable type. The object supports both integer- and label-based indexing and provides a host...
Read more >
Series — pandas 1.5.2 documentation
Return number of unique elements in the object. Series.is_unique. Return boolean if values in the object are unique.
Read more >
Essential basic functionality — pandas 1.5.2 documentation
pandas objects ( Index , Series , DataFrame ) can be thought of as containers for arrays, which hold the actual data and...
Read more >
pandas.Series.replace — pandas 1.5.2 documentation
For a DataFrame a dict can specify that different values should be replaced in different columns. For example, {'a': 1, 'b': 'z'} looks...
Read more >
pandas.Series.sample — pandas 1.5.2 documentation
If called on a DataFrame, will accept the name of a column when axis = 0. Unless weights are a Series, weights must...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found