question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Confusing (possibly buggy) IntervalIndex behavior

See original GitHub issue
screen shot 2017-05-09 at 10 03 25 pm

In the above, I have a region that I’m querying for with a partially overlapping interval. The query succeeds when the interval is partially overlapping until it doesn’t, throwing the key error:

KeyError                                  Traceback (most recent call last)
/Users/alex/Documents/GarNet/venv/lib/python3.6/site-packages/pandas/core/indexing.py in _has_valid_type(self, key, axis)
   1433                 if not ax.contains(key):
-> 1434                     error()
   1435             except TypeError as e:

/Users/alex/Documents/GarNet/venv/lib/python3.6/site-packages/pandas/core/indexing.py in error()
   1428                 raise KeyError("the label [%s] is not in the [%s]" %
-> 1429                                (key, self.obj._get_axis_name(axis)))
   1430 

KeyError: 'the label [(5409951, 5409965]] is not in the [index]'

I think this is particularly confusing because there doesn’t seem to be any prominent difference between the locs that succeed and the loc that fails as far as I can tell. I know we had discussed loc’s behavior in this context but I’m not sure we came to a conclusion.

By the way, my larger question is about how to find intersections between two IntervalIndex. It seems like the find_intersections function didn’t make it into this release @jreback ? Let me know! =]

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:37 (36 by maintainers)

github_iconTop GitHub Comments

2reactions
buyologycommented, May 18, 2017

chiming in on this, as we are heavy users of postgres range types and range operators as a powerful abstraction for time series data

as already been mentioned, the key verbs are contains and overlaps both on element and range level and in both directions:

examples from the postgres docs:

Operator Description Example Result
@> contains range int4range(2,4) @> int4range(2,3) t
@> contains element '[2011-01-01,2011-03-01)'::tsrange @> '2011-01-10'::timestamp t
<@ range is contained by int4range(2,4) <@ int4range(1,7) t
<@ element is contained by 42 <@ int4range(1,7) f
&& overlap (have points in common) int8range(3,7) && int8range(4,12) t
<< strictly left of int8range(1,10) << int8range(100,110) t
-|- is adjacent to numrange(1.1,2.2) -|- numrange(2.2,3.3) t

now that we have Intervals in pandas (very grateful for bringing that feature @jreback!) I have already tinkered around with some mappers for going between Postgres and pandas — maybe that is too db-specific but def have a great interest in seeing more Interval type functionality in Pandas and helping out with this

1reaction
jrebackcommented, Apr 11, 2018

came across this library: https://github.com/AlexandreDecan/python-intervals

looks to have some interesting interval semantics

cc @jschendel

Read more comments on GitHub >

github_iconTop Results From Across the Web

What's New — pandas 0.20.2 documentation - PyData |
Possible incompatibility for HDF5 formats created with pandas < 0.13.0 ... These indexing behaviors of the IntervalIndex are provisional and may change in...
Read more >
Innovation in Governmental Structures - JSTOR
Research into the causes of innovation adoption has developed a body of hypotheses which appear to contain contradictory elements.
Read more >
ML070600389.pdf - Nuclear Regulatory Commission
to cerium if the cerium compounds in fact behave as class W aerosols. The categorizing of the aerosol clearance from the pulmonary region....
Read more >
New World Mennonite Low German - UC Berkeley
established and it is unclear whether they are features of Modern Low German or ... some form of shunning behavior typical of excommunication,...
Read more >
What's New — pandas 0.23.4 documentation
To retain the current behavior, convert the 'datetime.date' to a datetime with 'pd. ... IntervalIndex.astype now supports conversions between subtypes when ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found