Setting host.hostname mapping to static

Elastic Stack Beats
1

ECE 2.3
Metricbeat 7.0.0 & 7.5.0

I need some guidance on how to stop the dynamic templating for host.hostname of our metricbeats. We've got a third party product that can only read data from the host.hostname field if the mapping is the same as metricbeat 7.0.0. From the dev tools I got part of the host.hostname mapping and as you can see its just has a "type" of "keyword".

        "host" : {
          "properties" : {
            "architecture" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            "containerized" : {
              "type" : "boolean"
            },
            "geo" : {
              "properties" : {
                "city_name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "continent_name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "country_iso_code" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "country_name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "location" : {
                  "type" : "geo_point"
                },
                "name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "region_iso_code" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                },
                "region_name" : {
                  "type" : "keyword",
                  "ignore_above" : 1024
                }
              }
            },
            "group" : {
              "type" : "keyword"
            },
            "hostname" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },

I've tried to load the 7.0.0 template manually and apply it to metricbeat-7.5.0 so that the application can read the data. What I'm having problems with is that the host.hostname field is being mapped as a text field and keyword field it appears which is preventing the app from reading the data correctly. How do I set the mapping up so that hostname is just type keyword and not text.

      "host": {
        "properties": {
          "architecture": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "containerized": {
            "type": "boolean"
          },
          "group": {
            "type": "keyword"
          },
          "hostname": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
2

To update this a little bit both of the templates have the hostname section setup as

},
          "hostname": {
            "ignore_above": 1024,
            "type": "keyword"
          },

The problem seems to be when metricbeat 7.5.0 starts sending data to the index it automatically changes it to

 "hostname": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },

Which is why the program can't read it.

3

Seems like something is adding the fields portion to every field but I'm not sure where its coming from.

{
 "mapping": {
  "properties": {
   "@timestamp": {
    "type": "date"
   },
   "@version": {
    "type": "text",
    "fields": {
     "keyword": {
      "type": "keyword",
      "ignore_above": 256
     }
    }
   },
   "agent": {
    "properties": {
     "ephemeral_id": {
      "type": "text",
      "fields": {
       "keyword": {
        "type": "keyword",
        "ignore_above": 256
       }
      }
     },
     "hostname": {
      "type": "text",
      "fields": {
       "keyword": {
        "type": "keyword",
        "ignore_above": 256
       }
      }
     }....
4

The problem seems to be using an alias for metricbeat. If I don't create an alias for metricbeat-7.5.0 the mapping is correct. If I apply an alias to an index I the mapping adds the extra field.

What I've done is stop all reporting 7.5 metricbeats and delete the metricbeat-7.5.0-2020.01.03 index. From there I ran the command below to setup an index with date math that has the alias metricbeat-7.5.0.

PUT /%3Cmetricbeat-7.5.0-%7Bnow%2Fd%7D-000001%3E
{
 "aliases": {
  "metricbeat-7.5.0": {
   "is_write_index": true
 }
 }
}

When this setup occurs the mapping changes which is causing the issue for our 3rd party program.

5

I'm running into a second issue with this. What seems to be happening and what I can't figure out is why a template for metricbeat-7.3.0 keeps coming up when the only beats reporting to this cluster are either 7.0 or 7.5.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.