# Beat templates do not stick to ECS?

**URL:** <https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064>\
**Category:** Beats\
**Tags:** filebeat, winlogbeat, auditbeat\
**Created:** [July 14, 2020, 6:37am UTC](https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064 "2020-07-14T06:37:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![BBQigniter](https://avatars.discourse-cdn.com/v4/letter/b/3da27b/32.png) [@BBQigniter](https://discuss.elastic.co/u/BBQigniter)\
**Post date:** [July 14, 2020, 6:37am UTC](https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064/1 "2020-07-14T06:37:15Z")

</div>

Hello,  
am I seeing this correctly that the Beat-templates sometimes do not stick to the ECS? For example I wanted to prepare a mapping template for our ELK-Stack and so I merged auditbeat, filebeat and winlogbeat mappings together via a short python script and wanted to combine it with the ECS (because here everything goes through some logstash-buffer/-filter combination) so all possible fields are mapped in proper way. My plan is to create one index for beats and one for other stuff that is mostly syslog-filtered stuff.  
Unfortunately the mapping gets a mess by just merging the "mappings", because some fields from the Beats use alias-fields and some not. Also there are fields that have the same name but in one beat-template there are also some nested fields below.

For example the "source"-field. As far as I can see in Filebeat it's a single field, in Auditbeat and ECS-template it's a nested field.

```auto
    # filebeat              
              "source": {
                "ignore_above": 1024,
                "type": "keyword"
              },
              
    # ecs
      "source": {
        "properties": {
          "address": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "as": {
            "properties": {
              "number": {
                "type": "long"
              },
              "organization": {
                "properties": {
                  "name": {
                    "fields": {
                      "text": {
                        "norms": false,
                        "type": "text"
                      }
                    },
                    "ignore_above": 1024,
                    "type": "keyword"
                  }
                }
              }
            }
          },

    # auditbeat
      "source": {
        "properties": {
          "address": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          "as": {
            "properties": {
              "number": {
                "type": "long"
              },
              "organization": {
                "properties": {
                  "name": {
                    "fields": {
                      "text": {
                        "norms": false,
                        "type": "text"
                      }
                    },
                    "ignore_above": 1024,
                    "type": "keyword"
                  }
                }
              }
            }
          },
          "bytes": {
            "type": "long"
          },
          "domain": {
            "ignore_above": 1024,
            "type": "keyword"
          },
          [...shortened...]

```

Also the fields that use aliases give me a headache

```auto
    # filebeat & winlogbeat            
        "container": {
          "properties": {
            "id": {
              "path": "docker.container.id",
              "type": "alias"
            },
            "image": {
              "properties": {
                "name": {
                  "path": "docker.container.image",
                  "type": "alias"
                }
              }
            },
            "name": {
              "path": "docker.container.name",
              "type": "alias"
            }
          }
        },
              
    # ecs
        "container": {
          "properties": {
            "id": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "image": {
              "properties": {
                "name": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "tag": {
                  "ignore_above": 1024,
                  "type": "keyword"
                }
              }
            },
            "labels": {
              "type": "object"
            },
            "name": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "runtime": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },

```

It would be awesome if that gets fixed some day so everything really keeps to ECS. Also ECS uses:

```auto
      "mappings": {
         "_doc": { 

```

and beat-templates

```auto
      "mappings": {
         "doc": { 

```

which one should you use?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 14, 2020, 7:21am UTC](https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064/2 "2020-07-14T07:21:14Z")

</div>

You should really keep these in their own indices. Simply dumping them all into the single one will likely lead to a mapping explosion, which is not a great thing to deal with.

---

<div class="post-metadata">

**Author:** ![BBQigniter](https://avatars.discourse-cdn.com/v4/letter/b/3da27b/32.png) [@BBQigniter](https://discuss.elastic.co/u/BBQigniter)\
**Post date:** [July 14, 2020, 8:08am UTC](https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064/3 "2020-07-14T08:08:37Z")

</div>

ok. thx for clarification. 🙂

but imho if elastic is proposing to use ECS they really should fix this so the mappings are really everywhere the same - justmy2cents

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 14, 2020, 8:10am UTC](https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064/4 "2020-07-14T08:10:20Z")

</div>

Oh for sure, there's probably a few discrepancies that we are yet to sort out.  
If you find some, please create a GitHub issue so we can get them resolved.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 11, 2020, 10:10am UTC](https://discuss.elastic.co/t/beat-templates-do-not-stick-to-ecs/241064/5 "2020-08-11T10:10:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.