# Bug? Search (specific) query doesn't return documents that does exist!

**URL:** <https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352>\
**Category:** Elasticsearch\
**Created:** [August 31, 2015, 12:29pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352 "2015-08-31T12:29:23Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nir\_Reuveny](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@Nir\_Reuveny](https://discuss.elastic.co/u/Nir_Reuveny)\
**Post date:** [August 31, 2015, 12:29pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/1 "2015-08-31T12:29:23Z")

</div>

Hi,

We have ES 1.7 cluster with daily logstash indexes.  
We've noticed a major issue which we can't explain... when searching (or running term aggr) on a specific field we need it doesn't return any documents, although we do have many documents with that value or any other value in that index!  
This doesn't happen on all 'daily' indexes... just on some...

See below examples that shows the problem:

```auto
http://kibana:9200/logstash-2015.08.29/_search
{
  "size": 200,
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "_type:\"record\" AND d_id_pre_1:\"c\"",
          "analyze_wildcard": true
        }
      }
    }
  },
  "fields": [
    "d_id_pre_1"
  ]
}

```

Results:

```auto
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}

```

* * *

Doing the search on the same index without filtering the value of the d\_id\_pre\_1, you can see that there are documents with "c" as the value of the d\_id\_pre\_1 field!

```auto
{
  "size": 5,
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "_type:\"record\"",
          "analyze_wildcard": true
        }
      }
    }
  },
  "fields": [
    "d_id_pre_1"
  ]
}

```

Result:

```auto
{
  "took": 52,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "hits": {
    "total": 4619845,
    "max_score": 1,
    "hits": [
      {
        "_index": "logstash-2015.08.29",
        "_type": "record",
        "_id": "b1ad227e8a8b3e3044c6822c55e1399b",
        "_score": 1,
        "fields": {
          "d_id_pre_1": [
            "2"
          ]
        }
      },
      {
        "_index": "logstash-2015.08.29",
        "_type": "record",
        "_id": "e0dcf0c4ea01b2d0b98a247955e1399b",
        "_score": 1,
        "fields": {
          "d_id_pre_1": [
            "2"
          ]
        }
      },
      {
        "_index": "logstash-2015.08.29",
        "_type": "record",
        "_id": "e371b878878935a4ec7d7c3355e1399c",
        "_score": 1,
        "fields": {
          "d_id_pre_1": [
            "c"
          ]
        }
      },
      {
        "_index": "logstash-2015.08.29",
        "_type": "record",
        "_id": "c8ac1a8e23c73175e0660fc655e1399c",
        "_score": 1,
        "fields": {
          "d_id_pre_1": [
            "1"
          ]
        }
      },
      {
        "_index": "logstash-2015.08.29",
        "_type": "record",
        "_id": "6479d993fe922d39dba61c9f55e1399c",
        "_score": 1,
        "fields": {
          "d_id_pre_1": [
            "7"
          ]
        }
      }
    ]
  }
}

```

* * *

This problem is happening on some of the daily indexes, not all of them... which makes this issue even more odd...  
You can see this aggr query/results that shows this:

```auto
http://kibana:9200/logstash-2015.08.31,logstash-2015.08.30,logstash-2015.08.29,logstash-2015.08.28/_search
{
  "size": 0,
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "query": "_type:\"record\" AND d_id_pre_1:\"c\"",
          "analyze_wildcard": true
        }
      }
    }
  },
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "1d",
        "min_doc_count": 0
      }
    }
  }
}

```

Result:

```auto
{
  "took": 25,
  "timed_out": false,
  "_shards": {
    "total": 8,
    "successful": 8,
    "failed": 0
  },
  "hits": {
    "total": 269432,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "2": {
      "buckets": [
        {
          "key_as_string": "2015-08-28T00:00:00.000Z",
          "key": 1440720000000,
          "doc_count": 140945
        },
        {
          "key_as_string": "2015-08-29T00:00:00.000Z",
          "key": 1440806400000,
          "doc_count": 0
        },
        {
          "key_as_string": "2015-08-30T00:00:00.000Z",
          "key": 1440892800000,
          "doc_count": 0
        },
        {
          "key_as_string": "2015-08-31T00:00:00.000Z",
          "key": 1440979200000,
          "doc_count": 128487
        }
      ]
    }
  }
}

```

* * *

Ideas?? seems like a huge bug at this point... as I can't find any good reason for this behavior...

Thanks!

Nir.

---

<div class="post-metadata">

**Author:** ![Nir\_Reuveny](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@Nir\_Reuveny](https://discuss.elastic.co/u/Nir_Reuveny)\
**Post date:** [September 3, 2015, 7:56am UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/2 "2015-09-03T07:56:14Z")

</div>

Anyone can help or have any ideas on this problem?

---

<div class="post-metadata">

**Author:** ![mikemccand](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mikemccand](https://discuss.elastic.co/u/mikemccand)\
**Post date:** [September 3, 2015, 8:19am UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/3 "2015-09-03T08:19:49Z")

</div>

How is the d\_id\_pre\_1 field indexed in the problematic daily index? Is it analyzed (which analyzer)?

Can you try removing the double-quotes around the query? This tells the query parser to make a phrase query, but (at least in this example) you have only one token (c) that you are trying to match ...

---

<div class="post-metadata">

**Author:** ![Nir\_Reuveny](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@Nir\_Reuveny](https://discuss.elastic.co/u/Nir_Reuveny)\
**Post date:** [September 3, 2015, 11:06am UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/4 "2015-09-03T11:06:57Z")

</div>

Hi Mike,

The field is set to 'not\_analyzed' in all daily indexes... (coming from the same template...)  
So AFAIK you need to search for the full text which I did...

"d\_id\_pre\_1" : {  
"index" : "not\_analyzed",  
"type" : "string"  
}

---

<div class="post-metadata">

**Author:** ![mikemccand](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mikemccand](https://discuss.elastic.co/u/mikemccand)\
**Post date:** [September 3, 2015, 12:56pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/5 "2015-09-03T12:56:09Z")

</div>

OK good, yes you must search for the full text.

Did you try the query without double quotes around c?

---

<div class="post-metadata">

**Author:** ![Nir\_Reuveny](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@Nir\_Reuveny](https://discuss.elastic.co/u/Nir_Reuveny)\
**Post date:** [September 3, 2015, 1:27pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/6 "2015-09-03T13:27:47Z")

</div>

Just did. it's the same result... again, the mappings are exactly the same on all indexes. but the search just doesn't 'work' on some of the indexes...  
Seems like a bug, no?

{  
"size": 200,  
"query": {  
"filtered": {  
"query": {  
"query\_string": {  
"query": "\_type:"record" AND d\_id\_pre\_1:c",  
"analyze\_wildcard": true  
}  
}  
}  
}  
}

{  
"took": 2,  
"timed\_out": false,  
"\_shards": {  
"total": 2,  
"successful": 2,  
"failed": 0  
},  
"hits": {  
"total": 0,  
"max\_score": null,  
"hits": []  
}  
}

---

<div class="post-metadata">

**Author:** ![mikemccand](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mikemccand](https://discuss.elastic.co/u/mikemccand)\
**Post date:** [September 3, 2015, 5:05pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/7 "2015-09-03T17:05:52Z")

</div>

Yeah maybe a bug ... can you simplify it down to a small case?

E.g. remove the type:"record" part, remove the analyze\_wildcard, use a straight query (not filtered)?

---

<div class="post-metadata">

**Author:** ![Nir\_Reuveny](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@Nir\_Reuveny](https://discuss.elastic.co/u/Nir_Reuveny)\
**Post date:** [September 3, 2015, 6:54pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/8 "2015-09-03T18:54:14Z")

</div>

Mike,

I've tried that. if I remove the 'type' filter, those 'problematic' indexes does return results but only for other document types and not to the main type we use in 90% of our logs ('record')  
I've also tried to run term aggr... same behavior! it shows really small numbers on the problematic indexes as it doesn't find (or ignore in some way) most of the documents (the ones with 'record' type)...  
But on the 'good' indexes it shows the very high numbers for each bucket in the aggr...

{  
"size": 0,  
"query": {  
"filtered": {  
"query": {  
"query\_string": {  
"query": "\*"  
}  
}  
}  
},  
"aggs": {  
"3": {  
"terms": {  
"field": "d\_id\_pre\_1",  
"size": 20  
}  
}  
}  
}

```
    {
      "key": "0",
      "doc_count": 30
    },
    {
      "key": "1",
      "doc_count": 30
    },
    {
      "key": "6",
      "doc_count": 28
    },
    {
      "key": "c",
      "doc_count": 26
    },
    {
      "key": "f",
      "doc_count": 24
    }
```

---

<div class="post-metadata">

**Author:** ![Nir\_Reuveny](https://avatars.discourse-cdn.com/v4/letter/n/ebca7d/32.png) [@Nir\_Reuveny](https://discuss.elastic.co/u/Nir_Reuveny)\
**Post date:** [September 7, 2015, 6:50am UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/9 "2015-09-07T06:50:37Z")

</div>

Bumping this problem... anyone?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:51pm UTC](https://discuss.elastic.co/t/bug-search-specific-query-doesnt-return-documents-that-does-exist/28352/10 "2017-07-05T23:51:50Z")

</div>