# Duplicating message parts in Elastic dashboard

**URL:** <https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791>\
**Category:** Logstash\
**Created:** [October 30, 2019, 5:27am UTC](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791 "2019-10-30T05:27:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kavindu\_manaram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavindu_manaram/32/56614_2.png) [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\
**Post date:** [October 30, 2019, 5:27am UTC](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791/1 "2019-10-30T05:27:17Z")

</div>

I had shipped log to ES with the integration of filebeat and logstash.  
This is my logstash.conf

```
# Read input from filebeat by listening to port 5044 on which filebeat will send the data
input {
    beats {
	    type => "test"
        port => "5044"
    }
}
 
filter {
  #If log line contains tab character followed by 'at' then we will tag that entry as stacktrace
  if [fields][level] == "info" {
    grok {
      match => {"message" => ["(?m)%{TIMESTAMP_ISO8601:timestamp} - %{NOTSPACE:orgCode} - %{GREEDYDATA:message}"]}
	  #break_on_match => true
      add_tag => ["cms-info"]
	  tag_on_failure => ["_grokparsefailure-cms-info"]
    }
 }
output {
   
  stdout {
    codec => rubydebug
  }
 
  # Sending properly parsed log events to elasticsearch
    elasticsearch {
    hosts => ["https://xxxxxxxx.us-east-1.aws.found.io:443"]
    index => "%{[fields][service-name]}-%{[fields][env]}-%{[fields][application]}-%{[fields][level]}-%{+yyyy.MM.dd}"
    user => "elastic"
    password => "xxxxxx"
  }
}

```

This is my log message

`"2019-10-29 21:57:16.9884 - yyyyy - Generating [item settings cloudfront item links] for asset : 137909"`

But in my ES board showsed duplicating message parts. like following

 ![1](https://us1.discourse-cdn.com/elastic/original/3X/6/c/6c5474abf62db4d15fd4b50d582daa588d7a487b.png)  
How can I solve this?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 30, 2019, 5:44am UTC](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791/2 "2019-10-30T05:44:26Z")

</div>

You have not specified the [overwrite parameter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-overwrite), so the rest of the message is appended to the message field instead of replacing it.

> [@kavindu\_manaram](#):
>
> index =\> "%{[fields][service-name]}-%{[fields][env]}-%{[fields][application]}-%{[fields][level]}-%{+yyyy.MM.dd}"

On an unrelated note, this looks like a pattern that could result in a lot of very small indices and shards. This can cause performance and stability problems down the line, so I would recommend you reconsider this approach. See [this blog post](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster) for details.

---

<div class="post-metadata">

**Author:** ![kavindu\_manaram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavindu_manaram/32/56614_2.png) [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\
**Post date:** [October 30, 2019, 6:13am UTC](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791/3 "2019-10-30T06:13:24Z")

</div>

Thanks it works !!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 27, 2019, 6:13am UTC](https://discuss.elastic.co/t/duplicating-message-parts-in-elastic-dashboard/205791/4 "2019-11-27T06:13:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.