# How make multi shipping without duplicates **URL:** \ **Category:** Logstash\ **Created:** [August 18, 2015, 12:26pm UTC](https://discuss.elastic.co/t/how-make-multi-shipping-without-duplicates/27583 "2015-08-18T12:26:55Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![inceptor](https://avatars.discourse-cdn.com/v4/letter/i/bb73d2/32.png) [@inceptor](https://discuss.elastic.co/u/inceptor)\ **Post date:** [August 18, 2015, 12:26pm UTC](https://discuss.elastic.co/t/how-make-multi-shipping-without-duplicates/27583/1 "2015-08-18T12:26:55Z")
Hello everybody, I was try it some examples (architectural) from the documentation : [https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html#deploying-scaling](https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html#deploying-scaling) But I have now a problem : The multi shipping from one or many source creates duplicates logs. Of course, if I have 3 shpping binding one same source, i will have 3 same logs. Has you can see below, an example of the infrastructure I trying to make: [https://www.elastic.co/guide/en/logstash/current/static/images/deploy\_7.png](https://www.elastic.co/guide/en/logstash/current/static/images/deploy_7.png) Now, I want to remove the duplicates logs from differente shipping (Have 1 log on the queue, not 3 same logs because I have 3 shipping on the same source). Someone can help me ? Thank you to read me and thank you a lot for the reponse(s) ! 😀 ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 18, 2015, 11:35pm UTC](https://discuss.elastic.co/t/how-make-multi-shipping-without-duplicates/27583/2 "2015-08-18T23:35:30Z")
> [@inceptor](#): > > The multi shipping from one or many source creates duplicates logs. You have multiple LS shippers reading a single source? ---
**Author:** ![inceptor](https://avatars.discourse-cdn.com/v4/letter/i/bb73d2/32.png) [@inceptor](https://discuss.elastic.co/u/inceptor)\ **Post date:** [August 19, 2015, 9:16am UTC](https://discuss.elastic.co/t/how-make-multi-shipping-without-duplicates/27583/3 "2015-08-19T09:16:30Z")
Hello warkolm and thank you to reply me, Yes, I have multiple LS shippers reading a single source (avoiding the Single Point Of Failure). For exemple, I have 3 LS shippers reading a same channel on one IRC. The result : I have 3 duplicates messages. Here somes pictures : The configuration of LS shippers number 2 (the other has the same idea, just user/real/nick change) : ![](https://us1.discourse-cdn.com/elastic/original/2X/2/2b99d566017ba17a1157937ca45f6c43d3720a87.png) The result (duplicates logs): ![](https://us1.discourse-cdn.com/elastic/original/2X/8/84ce9fc3d32c1d7e49b6c6d990810694e016a26a.png) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 21, 2015, 9:39am UTC](https://discuss.elastic.co/t/how-make-multi-shipping-without-duplicates/27583/4 "2015-08-21T09:39:02Z")
> [@inceptor](#): > > For exemple, I have 3 LS shippers reading a same channel on one IRC. The result : I have 3 duplicates messages. Well that'd be why, the IRC input is pretty dumb in that regards. Take a look at [https://www.elastic.co/guide/en/logstash/current/plugins-filters-checksum.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-checksum.html), it may help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:31am UTC](https://discuss.elastic.co/t/how-make-multi-shipping-without-duplicates/27583/5 "2017-07-06T05:31:21Z")
Closed on July 6, 2017, 5:31am UTC