ES BUG? Filter excluding "-" still brings results with dash "-"

[cawoodm (Marc)]

We have an index with documents which have the value "-" in a field "referrer". However when we try to exclude these values in Kibana, the documents are not filtered out. Digging deeper we see that ElasticSearch does not respect the query:

The query NOT referrer: \"-\" returns a document with referrer = -.

[Christian_Dahlqvist]

What is the mapping for the referrer field?

[cawoodm (Marc)]

...
"referrer": {
    "type": "text"
}
...

[dadoonet (David Pilato)]

Such a field is applying a standard analyzer to the text.
Use the _analyze API to see how your text is transformed. Most likely, nothing is indexed.

[cawoodm (Marc)]

I'm confused - the field is text (and not analyzed?) so I can't filter on it's existence?

[dadoonet (David Pilato)]

NOT referrer: \"-\" is not querying on existence of the field but comparing its content. Here you indexed no content. And you are searching for nothing.

Here is an example that shows this:

DELETE test 
PUT test/_doc/1
{
  "foo": "bar"
}
PUT test/_doc/2
{
  "foo": "-"
}
GET test/_search
{
  "query": {
    "query_string": {
      "query": "NOT foo:\"-\""
    }
  }
}

This gives nothing.

But:

GET test/_search
{
  "query": {
    "query_string": {
      "query": "NOT foo.keyword:\"-\""
    }
  }
}

Gives:

{
  "took" : 20,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "foo" : "bar"
        }
      }
    ]
  }
}