# Add tag to filebeat modules

**URL:** <https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 3, 2021, 10:42am UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314 "2021-11-03T10:42:36Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\
**Post date:** [November 3, 2021, 10:42am UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/1 "2021-11-03T10:42:37Z")

</div>

hello!

I am trying to send the logs with the filebeat mysql module to my logstash and it already obtains the pipeline and does the parsing correctly.

How can I differentiate both inputs ( Slowlog and errror ) to apply different filters to them?

for example:

```auto
  if [log][file][path] == "/var/lib/mysql/g99dnap797-slow.log" {
    elasticsearch {
      hosts => ["https://x:9200","https://y:9200","https://z:9200"]
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-tecnico-mysql-%{[agent][hostname]}-%{+YYYY.MM.dd}"
      ssl => true
      manage_template => false
      user => "elastic"
      password => "xxxxxxx"
      pipeline => "filebeat-7.13.2-mysql-slowlog-pipeline"
    }
  }

```

the module:

```auto
- module: mysql
  # Error logs
  error:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

  # Slow logs
  slowlog:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [November 3, 2021, 11:17pm UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/2 "2021-11-03T23:17:47Z")

</div>

There are other fields in `@metadata` that u may be able to use our should be able to use `event.dataset` which in this case would be `mysql.error` and `mysql.slowlog`

---

<div class="post-metadata">

**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\
**Post date:** [November 4, 2021, 3:21pm UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/3 "2021-11-04T15:21:17Z")

</div>

first of all , thanks for answering!

I'm trying in this way but it doesnt work.

After this I have an else condition and all the data is skipping the dataset and passing through the else, why?

```auto
  if [event][dataset] == "apache.error" {
    elasticsearch {
      hosts => ["https://xxx:9200"]
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-tecnico-apache-%{[agent][hostname]}-%{+YYYY.MM.dd}"
      ssl => true
      manage_template => false
      user => "elastic"
      password => "xxxxxxxt"
      pipeline => "filebeat-7.13.2-apache-error-pipeline"
    }
  }

  if [event][dataset] == "mysql.error" {
    elasticsearch {
      hosts => ["https://xxxxx:9200"]
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-tecnico-mysql-%{[agent][hostname]}-%{+YYYY.MM.dd}"
      ssl => true
      manage_template => false
      user => "elastic"
      password => "xxxxxxxxxx"
      pipeline => "filebeat-7.13.2-mysql-error-pipeline"
    }
  }

```

thanks you!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 5, 2021, 12:55am UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/4 "2021-11-05T00:55:50Z")

</div>

So I think you can do this simpler something like

In the `filter` block do the if / else on the `event.data` type and set a field or tag with part of the index name like `index-subtype`

then your output could look something like

```auto
output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "http://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{index-subtype}"
      pipeline => "%{[@metadata][pipeline]}" 
      user => "elastic"
      password => "secret"
    }
  } else {
    elasticsearch {
      hosts => "http://localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{index-subtype}"
      user => "elastic"
      password => "secret"
    }
  }
}

```

Unclear why you want to separate by hostname... that often leads to many many indices with small shard which leads to over sharding... The `host.name` and `agent.hostname` is in every event and can be easily filtered ... just a thought

---

<div class="post-metadata">

**Author:** ![rodri.gz](https://avatars.discourse-cdn.com/v4/letter/r/aca169/32.png) [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\
**Post date:** [November 5, 2021, 7:35am UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/5 "2021-11-05T07:35:01Z")

</div>

ok, I'll try it that way.

Regarding the name of the indexes, we put the hostname because it is a cluster with quite specific data. Anyway, could you perform a reindex with a date variable so that it just removes the hostname?

for example:

```auto
POST _reindex
{
  "source": {
    "index": "filebeat-7.13.2-apache*"
  },
  "dest": {
    "index": "filebeat-7.13.2-apache-%{[@timestamp]}"
  }
}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 5, 2021, 4:53pm UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/6 "2021-11-05T16:53:46Z")

</div>

That is not going to work that destination index will be the literal of what you defined... and of course timestamp go down to the ms so you would probably not want to do that...

You would probably want something like ....

```auto
POST _reindex
{
  "source": {
    "index": "filebeat-7.13.2-apache-*-2021.11.*"
  },
  "dest": {
    "index": "filebeat-7.13.2-apache-2021.11"
  }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 3, 2021, 6:54pm UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314/7 "2021-12-03T18:54:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.