# Remove duplicates
**URL:** https://discuss.elastic.co/t/remove-duplicates/95135
**Category:** Logstash
**Created:** [July 31, 2017, 7:43am UTC](https://discuss.elastic.co/t/remove-duplicates/95135 "2017-07-31T07:43:42Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@raja\_gopal](https://discuss.elastic.co/u/raja_gopal)
#### Post date: [July 31, 2017, 7:43am UTC](https://discuss.elastic.co/t/remove-duplicates/95135/1 "2017-07-31T07:43:42Z")
I m using logstash to filter logs from jenkins server. I pull out the build.xml files and the problem is the same file is created as two events in elasticsearch. What happens is when the jenkins starts the build the build.xml file gets updated and it is pushed to logstash by filebeat and the file is not completed. So when the next time filebeat checks the file the file is updated with a single line and gets pushed again. So how can i avoid this conflict ?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 28, 2017, 7:44am UTC](https://discuss.elastic.co/t/remove-duplicates/95135/2 "2017-08-28T07:44:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.