ES Query returns results that doesn't not fit the query

[jac1241 (jacob.amerzaev@verint.com)]

Hi all,

I've stumbled upon weird behavior in ES, where it returns results that does not match the query.
We are on ES 2.4 with tribe, and i was wondering if anyone encountered such behavior.

Query

{
  "query": {
    "bool": {
      "must": [
        {
          "exists": {
            "field": "header.create_time"
          }
        }
      ],
      "must_not": [],
      "should": [
        {
          "term": {
            "o.dst.name": "10.61.248.170"
          }
        },
        {
          "term": {
            "o.sources.name": "10.61.248.170"
          }
        },
        {
          "term": {
            "o.src.name": "10.61.248.170"
          }
        }
      ]
    }
  },
  "sort": {
    "header.create_time": "desc"
  },
  "size": 1
}

Result (I've hidden some info to make it readable)

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 48,
    "successful": 48,
    "failed": 0
  },
  "hits": {
    "total": 18889,
    "max_score": null,
    "hits": [
      {
        "_index": "someindex",
        "_type": "sometime",
        "_id": "6053119379950236520",
        "_score": null,
        "_source": {
          "o": {
            "src": {
              "name": "0.0.0.0"
            },
            "dst": {
              "name": ""
            }
          }
        },
        "sort": [
          1492611944340
        ]
      }
    ]
  }
}

Thanks,

[Ivan (Ivan Brusic)]

Does the document contain a valid header.create_time? I am assuming so
since there is a sort value.

The should clauses of the boolean query are optional and are used to
increase the score of a document. Any matching should clause will increase
the score. As long as the must clause matches, you will get a hit.

What you are probably want is for that should match to be actually another
bool query alongside the existing exist clause. And since you are using
only exist and term queries, you also probably want to use filter clauses
and avoid scoring altogether.

{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "header.create_time"
}
},
{
"bool": {
"should": [
{
"term": {
"o.dst.name": "10.61.248.170"
}
},
{
"term": {
"o.sources.name": "10.61.248.170"
}
},
{
"term": {
"o.src.name": "10.61.248.170"
}
}
]
}
}
]
}
}
}

[jac1241 (jacob.amerzaev@verint.com)]

Thank you for the reply.

I'm not sure i understand why the score would be increased and i get a hit if none of the should terms are matched in the results?

Edit
Moreover, I've inserted the bool under one filter (like in your example) , this gave me an exception.
I've added another bool and it worked.

{
  "query": {
    "bool": {
      "filter": {
        "bool": {
          "must": [
            {
              "exists": {
                "field": "header.create_time"
              }
            }
          ],
          "must_not": [
            
          ],
          "should": [
            {
              "term": {
                "o.dst.name": "10.61.248.170"
              }
            },
            {
              "term": {
                "o.sources.name": "10.61.248.170"
              }
            },
            {
              "term": {
                "o.src.name": "10.61.248.170"
              }
            }
          ]
        }
      }
    }
  },
  "sort": {
    "header.create_time": "desc"
  },
  "size": 1
}

This is different than the example provided in ES 2.4 changelog:
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/breaking_20_query_dsl_changes.html#_literal_filtered_literal_query_and_literal_query_literal_filter_deprecated

What am i missing here?

Thanks!