Date Range filter not working properly

Elastic Stack Elasticsearch
1

Hi,

I've been getting wrong results in range filters in python.
The code and mapping are as below:-

Code1:-

GET _search {
    "query": {
            "filtered": {
                 "filter": {
                     "and": [{
                         "range": { 
                             "ts": { 
                                 "gte": "2016-09-11",
                                 "lte": "2016-12-12" 
                             } 
                         } 
                     }] 
                 } 
             } 
         }
     }

Code2:-

GET _search {
    "query": {
            "filtered": {
                 "filter": {
                     "and": [{
                         "range": { 
                             "ts": { 
                                 "gte": "2016-09-11"
                             } 
                         } 
                     }] 
                 } 
             } 
         }
     }

Mapping:-

"ts" : { 
    "type" : "date", 
    "format" : "strict_date_optional_time||epoch_millis" 
}

Code1 gives NO output, even when there are hundreds of data points in that range.
Code2 gives data points from 2015 along side valid data points.
NOTE:- I've tried ALL the previous solutions on this discussion forum, so kindly guide me with a different one.

2

If you are using ES 5.0 and above ES version, "and" filter is not supported any longer, you have to use, bool "must" filter.

3

Hey,

it's amazing that you tried all the possible solutions in the forum (to be honest, I doubt it a little), but your post misses a lot of information in order to reproduce or guide you to a different one, see this nice post

  • Which Elasticsearch version are you using?
  • Which JVM version are you using?
  • Can you provide the full mapping of your index? (curl or console compatible)
  • Can you provide example documents, that you have indexed? (curl or console compatible) - this should include documents that do not match the first query and also documents that do match the second query even though they should not.
  • Why are you using an and filter, if you only have one condition? Can you omit it to simplify the example?
  • Can you also try to search for a specific index/type to minimize the problem and to ensure this also happens when only one index is involved?
  • On top of that it might make sense to include the python sample code in addition, as the sense snippets are not working they way you pasted them

Thank you!

--Alex

5

Tried already, nothing changes with "must". :disappointed:

6

Hi @spinscale ,
Thanks for an elaborate guidance.

  1. ES version we've tried are 5.1 and 2.3.
  2. JVM version is 8
  3. The "and" is there because there are other queries too which are all strings(names, zips, addresses and other details). We are good with other fields. Problem is persistent only with date range queries. Whether I add that "and" or remove it(when I test for just the date range), doesn't change anything.
  4. I can't try adding "new" index/types as I don't have access to server side mapping. But I've tried term queries on other indices, everything is working fine except for the "ts".
  5. Also, please note that "term" query is working for date. and it returns me data points of that particular date perfectly fine. Problem occurs only when I apply range.

NOTE:- I would have to take permissions for full codes, mappings and index. So, it would be really nice if you could help me with the info provided.

7

Hey,

can you please recreate a minimal example. That means you dont have to take anything from your live configuration, just parts of the mapping, parts of the document to recreate the issue. It's not helpful to just paste a mapping with hundreds of fields, if querying only one does not work.

Everything without data is just guesswork and it's hard to get down to the actual problem.

--Alex

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.