# Problem with duplicate data in field

**URL:** <https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598>\
**Category:** Logstash\
**Created:** [July 11, 2017, 7:56am UTC](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598 "2017-07-11T07:56:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ahongyun](https://avatars.discourse-cdn.com/v4/letter/a/c77e96/32.png) [@ahongyun](https://discuss.elastic.co/u/ahongyun)\
**Post date:** [July 11, 2017, 7:56am UTC](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598/1 "2017-07-11T07:56:13Z")

</div>

Hi all

I have an issue with the data that is passed into elasticsearch from logstash  
below is my logstash config.

> if [type] == "coreservices-fs" or "coreservices-net" {  
> grok {  
> match =\> { "message" =\> [  
> "^%{TIMESTAMP\_ISO8601:Time} [%{DATA:Thread}] %{GREEDYDATA:Logger} : %{LOGLEVEL:Severity} - %{GREEDYDATA:KEYMessage}",  
> "Process with pid=%{NOTSPACE:PID} %{GREEDYDATA:Info}\nService stopped at %{GREEDYDATA:StopTime} by",  
> "Start time %{GREEDYDATA:StartTime} by" ] } }  
> grok {  
> match =\> { "source" =\> "/appdir/%{NOTSPACE:Env\_Name}/logs/%{WORD:Server}.%{WORD}.%{WORD:NPID}" } } }

the data being seen on kibana are as follows:

 ![](https://us1.discourse-cdn.com/elastic/original/3X/8/a/8a0cea175dc5d189e90db6e57d4335b9716eb31f.PNG)

as you can see, for several fields, the data is repeated multiple times.

I cant seem to figure out why.

Any help would be greatly appreciated.

Thank you

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 12, 2017, 7:21pm UTC](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598/2 "2017-07-12T19:21:08Z")

</div>

Please show the full configuration. _All_ files in /etc/logstash/conf.d.

> if [type] == "coreservices-fs" or "coreservices-mxnet" {

This conditional expression doesn't mean what you think it means. See [Logstash sending same message to multiple indexes in ES - #2 by magnusbaeck](https://discuss.elastic.co/t/logstash-sending-same-message-to-multiple-indexes-in-es/92464/2).

---

<div class="post-metadata">

**Author:** ![ahongyun](https://avatars.discourse-cdn.com/v4/letter/a/c77e96/32.png) [@ahongyun](https://discuss.elastic.co/u/ahongyun)\
**Post date:** [July 13, 2017, 1:10am UTC](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598/3 "2017-07-13T01:10:45Z")

</div>

Perfect.

Thanks

This is great.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 10, 2017, 1:11am UTC](https://discuss.elastic.co/t/problem-with-duplicate-data-in-field/92598/4 "2017-08-10T01:11:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.