I posted here: stackoverflow but I think this forum is probably better suited to helping me solve my problem.
I basically have a postgres server generating logs like so:
My input/output is very simple:
path => '/pglog/postgresql-*.csv'
type => 'postgres_csv'
path => '/pglog/test_logstash.output'
But after 24 hours I'm seeing TONS of duplicate logs in /pglog/test_logstash.output
The sincedb files are indeed being updated, and there is only one agent running on this box.
Any ideas how I can trouble shoot this?
My guess is that postgres's logs are somehow confusing logstash, but I don't understand how I can fix this.