# Same message being sent twice

**URL:** <https://discuss.elastic.co/t/same-message-being-sent-twice/141893>\
**Category:** Logstash\
**Created:** [July 27, 2018, 7:36am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893 "2018-07-27T07:36:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![xShadow](https://avatars.discourse-cdn.com/v4/letter/x/b4bc9f/32.png) [@xShadow](https://discuss.elastic.co/u/xShadow)\
**Post date:** [July 27, 2018, 7:36am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893/1 "2018-07-27T07:36:36Z")

</div>

Hi,

I am not sure if I should place this topic under logstash or elasticsearch but I was testing the function of sending logs from logstash to elasticsearch and I noticed that the same message appears twice in the elasticsearch.

For example, I have a csv file where I add one line of message after starting logstash. The output of the configuration file points to my elasticsearch instance (single-node cluster), where I have also set an indexing template to set number of replication to 0. The output in my elasticsearch index shows the same message twice, but with different id. Am I supposed to set something to prevent this from happening?

Thank you!

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 27, 2018, 7:47am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893/2 "2018-07-27T07:47:39Z")

</div>

How exactly are you adding data to your file? A common mistake when testing is to edit the file using an editor, which typically creates a new file which is then renamed in the background. As this is a technically a new file with the same name, it will be reprocessed from the beginning. Make sure that you append to the existing file as that is what both Filebeat and Logstash expects.

---

<div class="post-metadata">

**Author:** ![xShadow](https://avatars.discourse-cdn.com/v4/letter/x/b4bc9f/32.png) [@xShadow](https://discuss.elastic.co/u/xShadow)\
**Post date:** [July 27, 2018, 7:56am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893/3 "2018-07-27T07:56:56Z")

</div>

Thanks for the reply. I was testing using a text editor (gedit) at first, but I have tested it using  
`echo test >> test.csv` after your reply and I'm still seeing two records in the index.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 27, 2018, 8:04am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893/4 "2018-07-27T08:04:50Z")

</div>

What does your config look like? How are you running Logstash? Are you restarting Logstash at any point or just leaving it running?

---

<div class="post-metadata">

**Author:** ![xShadow](https://avatars.discourse-cdn.com/v4/letter/x/b4bc9f/32.png) [@xShadow](https://discuss.elastic.co/u/xShadow)\
**Post date:** [July 27, 2018, 8:13am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893/5 "2018-07-27T08:13:47Z")

</div>

Below is the configuration file that I am using for testing purposes.

```
input {
    file {
        path => "/opt/logstash/log/test.csv"
    }
}
 
output {
    elasticsearch {
        hosts => ["x.x.x.x"]
        index => "testing"
    }
    stdout {}
}

```

I'm not sure what you mean by how am I running Logstash, but I specified the configuration file as shown below.  
`./logstash -f /path/to/config/test.conf`

I normally restart Logstash service after changing the configuration file in case of errors. i.e. I restarted the logstash service before doing what you mentioned earlier.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 24, 2018, 8:13am UTC](https://discuss.elastic.co/t/same-message-being-sent-twice/141893/6 "2018-08-24T08:13:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.