Hello, I have a setup with 2 syslog servers that get all logs from devices. Devices send to these 2 servers in parallel, but there is no sync between them. So the syslog servers have "almost" the same data, but I without guarantee (if one is down, the devices continue to log to the second). I cann…

[image] Efficient Duplicate Prevention for Event-Based Data in Elasticsearch Need to prevent duplicates in the Elastic Stack with minimal performance impact? In this blog we look at different options in Elasticsearch and provide some practical guidelines.

Discuss the Elastic Stack

Two filebeats with almost same data to send to Elastic

Elastic Stack Beats

kumarabhi (Abhishek) September 26, 2019, 5:48pm 2

Topic		Replies	Views
Duplicate events in filebeat + logstash + elasticsearch pipeline Logstash	2	1939	July 6, 2017
Syslogs, Beat, Redundancy and Architecture Beats	2	1384	July 6, 2016
Filebeat, Logstash, Elasticsearch robustness and duplicated documents Beats filebeat	11	4333	July 5, 2017
Deduplication in beats while sending logs to Logstash or Elasticsearch Beats	1	539	November 7, 2019
[SOLVED]Getting duplicate entries with Filebeat to Logstash Setup Beats filebeat	7	9997	December 5, 2018

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Two filebeats with almost same data to send to Elastic

Related topics