Keyword mapping is not working

[clrnd (Ezequiel Alvarez)]

Hi! First of all thanks to everyone in the community, so far I'm very pleased with the stack!

Now, I'm having an issue which is just driving me mad. I have this index template:

GET /_template/filebeat-6.5.4
{
  "filebeat-6.5.4" : {
    "order" : 1,
    "index_patterns" : [
      "filebeat-6.5.4-*"
    ],
    "settings" : {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "10000"
          }
        },
        "refresh_interval" : "5s",
        "number_of_routing_shards" : "30",
        "number_of_shards" : "1",
        "number_of_replicas" : "0"
      }
    },
    "mappings" : {
      "doc" : {
        "_meta" : {
          "version" : "6.5.4"
        },
        "date_detection" : false,
        "dynamic_templates" : [
          {
            "fields" : {
              "mapping" : {
                "type" : "keyword"
              },
              "match_mapping_type" : "string",
              "path_match" : "fields.*"
            }
          },
...

As you can see it has dynamic mappings for fields.*, which sounds okay. It's also the default template that filebeat creates so it sounds good.

Then, the indexes are created with the keyword mapping correctly afaik:

GET /filebeat-6.5.4-2019.02.19/_mapping/doc/field/fields.name
{
  "filebeat-6.5.4-2019.02.19" : {
    "mappings" : {
      "doc" : {
        "fields.name" : {
          "full_name" : "fields.name",
          "mapping" : {
            "name" : {
              "type" : "keyword"
            }
          }
        }
      }
    }
  }
}

BUT, as far as I can tell it isn't working.

If I do

GET /filebeat-6.5.4-2019.02.19/_search?q=fields.name:blah
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1176,
    "max_score" : 2.654241,
    "hits" : [...]

but if I do

GET /filebeat-6.5.4-2019.02.19/_search?q=fields.name.keyword:blah
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  }
}

So tl;dr: I have an index with a keyword mapping for a field, which is not working. How can I debug this issue? What's worst of all is that this used to work, I'm not sure what happened. I have different versions of filebeat in my servers so this is happening for 6.5.4 and 6.6.0, but afaik it should't be a problem.

Thanks!

[jpountz (Adrien Grand)]

This looks like the expected behavior to me. Maybe you are confused because the default mappings for strings are:

{
  "type": "text",
  "fields": {
    "keyword": {
      "type": "keyword",
      "ignore_above": 256
    }
  }
}

So if you create a field with name fields.name, you can either have access to its text representation via the fields.name field, or its keyword representation via the fields.name.keyword sub field.

With this template, you replaced this default mapping with

{
  "type": "keyword"
}

So fields.name is now of type keyword rather than text, and you don't have a fields.name.keyword subfield anymore.

[clrnd (Ezequiel Alvarez)]

Ohhhhhhhhhh that's it then. I'd say it's quite confusing as it is, one mapping gets the .keyword suffix, the other doesn't, and nothing indicates it? Would there have been a way of checking this myself?

Thank you so much!