# Duplicate postgres logs appearing in output

**URL:** <https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455>\
**Category:** Logstash\
**Created:** [March 25, 2016, 3:20pm UTC](https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455 "2016-03-25T15:20:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alexander\_Gray\_II](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexander_gray_ii/32/1669_2.png) [@Alexander\_Gray\_II](https://discuss.elastic.co/u/Alexander_Gray_II)\
**Post date:** [March 25, 2016, 3:20pm UTC](https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455/1 "2016-03-25T15:20:11Z")

</div>

I posted here: [stackoverflow](http://stackoverflow.com/questions/36221902/how-to-prevent-duplicate-logs-being-shipped-during-postgres-daily-logrotate) but I think this forum is probably better suited to helping me solve my problem.

I basically have a postgres server generating logs like so:  
postgresql-Sat.csv  
postgresql-Sun.csv  
postgresql-Mon.csv  
postgresql-Tue.csv  
postgresql-Wed.csv  
postgresql-Thu.csv  
postgresql-Fri.csv

My input/output is very simple:  
input {  
file {  
path =\> '/pglog/postgresql-\*.csv'  
type =\> 'postgres\_csv'  
}  
}  
output {  
file {  
path =\> '/pglog/test\_logstash.output'  
}  
}

But after 24 hours I'm seeing TONS of duplicate logs in /pglog/test\_logstash.output  
The sincedb files are indeed being updated, and there is only one agent running on this box.

Any ideas how I can trouble shoot this?

My guess is that postgres's logs are somehow confusing logstash, but I don't understand how I can fix this.

Any ideas?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 28, 2016, 8:19pm UTC](https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455/2 "2016-03-28T20:19:04Z")

</div>

If you start Logstash with `--verbose` it'll output sincedb stuff to the log. That might give clues about what's going on.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:05am UTC](https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455/3 "2017-07-06T05:05:07Z")

</div>

Closed on July 6, 2017, 5:05am UTC