# How to remove unwanted tags from logstash

**URL:** <https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760>\
**Category:** Logstash\
**Created:** [June 25, 2017, 4:36pm UTC](https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760 "2017-06-25T16:36:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![cazyfunlover](https://avatars.discourse-cdn.com/v4/letter/c/f08c70/32.png) [@cazyfunlover](https://discuss.elastic.co/u/cazyfunlover)\
**Post date:** [June 25, 2017, 4:36pm UTC](https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760/1 "2017-06-25T16:36:46Z")

</div>

Hi i wanted to remove duplicate tags like ("@version" ,"beat","hostname","name" ) , how can i do that ?

```
    {
                    "msg" => "Beginning Product data refresh",
        "timestamp_match" => "2017-06-25 12:16:13,904",
                  "level" => "INFO",
             "input_type" => "log",
                 "source" => "/archives/logs/tomcat7-8090/download.log",
                "message" => "[2017-06-25 12:16:13,904] :|: INFO :|: lvprdsndlbfe1.lv.jabodo.com :|: :|: :|: :|: :|: c.m.c.PeriodicProductDataRefresher :|: - Beginning Product data refresh",
                   "type" => "log",
                   "tags" => [
            [0] "multiline",
            [1] "beats_input_codec_multiline_applied"
        ],
             "@timestamp" => 2017-06-25T16:16:23.686Z,
               "@version" => "1",
                   "beat" => {
           "hostname" => "lvprdsndlbfe1",
             "name" => "lvprdsndlbfe1",
        "version" => "5.4.0"
        },
                  "class" => "c.m.c.PeriodicProductDataRefresher",
              "host_name" => "lvprdsndlbfe1.lv.jabodo.com"
    }
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 26, 2017, 11:15am UTC](https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760/2 "2017-06-26T11:15:03Z")

</div>

Use a mutate filter's `remove_field` option.

---

<div class="post-metadata">

**Author:** ![cazyfunlover](https://avatars.discourse-cdn.com/v4/letter/c/f08c70/32.png) [@cazyfunlover](https://discuss.elastic.co/u/cazyfunlover)\
**Post date:** [June 26, 2017, 1:36pm UTC](https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760/3 "2017-06-26T13:36:17Z")

</div>

I am already using mutate to remove unwanted tabs and new lines in the input .

Do i have to use mutate after parsing grok ( i am just concern on logstash server )

```
input {
    beats {
    client_inactivity_timeout => 86400
    port => 5044
    codec => multiline {
      pattern => "^\[%{TIMESTAMP_ISO8601}\]"
      negate => true
      what => previous
     }
    }
}
filter {
# csv {
# separator => ":|:"
# columns => ["Timestamp","Level","hostname","coidkey","Close","Volume", "Volume, "Currency","Weighted", "Price"]
# }
#}

  mutate {
    gsub => [
      # replace all forward slashes with underscore
      #"fieldname", "/", "_",
      # replace backslashes, question marks, hashes, and minuses
      # with a dot "."
      #"fieldname2", "[\\?#-]", "."
      "message", "\t", " ",
      "message", "\n", " "
    ]
  }
    grok {
    match => { "message" => "\[%{TIMESTAMP_ISO8601:timestamp_match}\]%{SPACE}\:\|\:%{SPACE}%{WORD:level}%{SPACE}\:\|\:%{SPACE}%{USERNAME:host_name}%{SPACE}\:\|\:%{SPACE}%{GREEDYDATA:coidkey}%{SPACE}\:\|\:%{SPACE}%{GREEDYDATA:clientinfo}%{SPACE}\:
\|\:%{SPACE}(%{IP:clientip})?%{SPACE}\:\|\:%{SPACE}%{GREEDYDATA:Url}%{SPACE}\:\|\:%{SPACE}%{JAVACLASS:class}%{SPACE}\:\|\:%{SPACE}%{USER:ident}%{SPACE}%{GREEDYDATA:msg}"} remove_field => ["ident","offset","name","version","host"]
   }
}
output {
    stdout { codec => rubydebug }

  if "_grokparsefailure" in [tags] {
    # write events that didn't match to a file
    file { "path" => "/tmp/grok_failures.txt" }
  } else{
     elasticsearch {
       hosts => "dfsyselastic.df.jabodo.com:9200"
       user => "UN"
       password => "PW"
       index => "vicinio-%{+YYYY.MM.dd}"
       document_type => "log"
     }
   }
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 26, 2017, 9:30pm UTC](https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760/4 "2017-06-26T21:30:04Z")

</div>

> I am already using mutate to remove unwanted tabs and new lines in the input .

That doesn't matter.

> Do i have to use mutate after parsing grok ( i am just concern on logstash server )

The fields you were talking about exist before the grok filter so it doesn't matter if you delete them before your after the grok filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 24, 2017, 9:30pm UTC](https://discuss.elastic.co/t/how-to-remove-unwanted-tags-from-logstash/90760/5 "2017-07-24T21:30:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.