Regression in date range query

There’s a potential regression when handling date range queries. The query we’re running is reported_date<date>:[2014-08-01 TO 2014-09-28]

When I run this against 0.10.0-SNAPSHOT I get

{
  "analyzer": "class org.apache.lucene.analysis.standard.StandardAnalyzer",
  "etag": "13769454259d",
  "fetch_duration": 1,
  "limit": 25,
  "plan": "NumericRangeQuery(1406808000000 TO 1411819200000 AS Long,boost=1.0)",
  "q": "reported_date:[1406808000000 TO 1411819200000]",
  "rows": [ ... ],
  "search_duration": 8,
  "skip": 0,
  "total_rows": 402
}

However running against 1.0.1 I get

{
  "analyzer": "class org.apache.lucene.analysis.standard.StandardAnalyzer",
  "etag": "138bdffdcec0",
  "fetch_duration": 3,
  "limit": 25,
  "plan": "NumericRangeQuery(1406808000000 TO 1411819200000 AS Long,boost=1.0)",
  "q": "reported_date:[1406808000000 TO 1411819200000]",
  "rows": [ ... ],
  "search_duration": 19,
  "skip": 0,
  "total_rows": 1
}

NB: “total_rows” is 1 instead of 402.

Futhermore if I increase the date range by one day so the query is reported_date<date>:[2014-08-01 TO 2014-09-29] I get “total_rows” of 0.

Interestingly the document returned from the second query has a reported_date of 2014-09-27T20:12:00+12:00 so it’s as if the query is searching for a period of one day before the end date regardless of what the start date is set to.

I’ve tried various forms of querying the date such as including a time component with no change.

The date field is configured as below

index: function(doc) {
  var ret = new Document();
  ...
  ret.add(new Date(doc.reported_date), { field: 'reported_date', type: 'date' });
  ...
  return ret;
}

[rnewson]

round two; I have a doc with the same date. If I query a range around it, I don’t find it. If I query a range around it (but much closer on one side), I do. Astonishing.

➜ cl-test git:(master) ✗ curl ‘127.0.0.1:5984/_fti/local/date/_design/foo/idx’ -d ‘debug=true&q=reported_date<long>:[1409526000000 TO 1412031600000]’ { “analyzer”: “class org.apache.lucene.analysis.standard.StandardAnalyzer”, “etag”: “24848b4c8f10”, “fetch_duration”: 0, “limit”: 25, “plan”: “NumericRangeQuery(1409526000000 TO 1412031600000 AS Long,boost=1.0)”, “q”: “reported_date:[1409526000000 TO 1412031600000]”, “rows”: [], “search_duration”: 0, “skip”: 0, “total_rows”: 0 }% ➜ cl-test git:(master) ✗ curl ‘127.0.0.1:5984/_fti/local/date/_design/foo/idx’ -d ‘debug=true&q=reported_date<long>:[1409526000000 TO 1411805520000]’ { “analyzer”: “class org.apache.lucene.analysis.standard.StandardAnalyzer”, “etag”: “24848b4c8f10”, “fetch_duration”: 0, “limit”: 25, “plan”: “NumericRangeQuery(1409526000000 TO 1411805520000 AS Long,boost=1.0)”, “q”: “reported_date:[1409526000000 TO 1411805520000]”, “rows”: [{ “id”: “foo”, “score”: 1 }], “search_duration”: 0, “skip”: 0, “total_rows”: 1 }%

[rnewson]

thanks for the bug report, that blew my mind for a bit there. 1.0.2 is out now with just this fix and one other.