From filebeat.template.json to fields.yml

Elastic Stack Beats
1

Hi,

We're about to upgrade filebeat 5.5.2 to filebeat 6.2.2
But then the default index template were changed from .json file to .yml file
My question is, how could I put the below json to yml format for us to be able to have a dynamic templates?

  "dynamic_templates": [
    {
      "ipAddress": {
        "match": "ipAddress",
        "mapping": {
          "type": "ip"
        }
      }
    }
  ]

Thank you!

2

According to your documentation:
By default Elasticsearch will map string fields as a text field with a sub keyword field. However if you are only indexing structured content and not interested in full text search, you can make Elasticsearch map your fields only as keywords.

But why am I seeing on my logs is the below dynamic_templates:

        "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": {
            "ignore_above": 1024,
            "type": "keyword"
3

Few things here:

  • You can specify a dynamic field by setting the field to type: object and then use object_type: ... to specify the type. For example:
    - name: fields
      type: object
      object_type: keyword
      description: >
        Contains user configurable fields.
  • The part you have above is a default we have in beats. Not sure if there is a good way to overwrite this. You can always use filebeat export template to get the raw template and then load it yourself. We are also planning to make loading from a template available again.
4

Hi ruflin,

That was a great help!
So what if I use filebeat export template, does it mean that it'll also be applied to the future logs or will it still be loaded the default template from filebeat 6.2.2?

Thanks.

5

If you export the template to a file and load it manually, make sure you disable automatic template loading in your Beat.

To simplify loading a template I just opened https://github.com/elastic/beats/pull/7039 It's not done yet, but we get there :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.