we decide to use Elasticsearch for our data store and handle different search scenarios with that. data is critical and should never lost. we should store it for 10 days. because data is time series we want to create index hourl. so different queries route into different indices. the other reason…

Best ways to prevent data lost in elastic

dadoonet (David Pilato) July 23, 2018, 8:40am 4

Yes. Most likely
Yes. That could be a good safeguard.

You can always consume the data from Kafka with Logstash and ask Logstash to store the raw data to whatever datastore you want (S3, HDFS, ...) and index the enriched data to Elasticsearch.

Topic		Replies	Views
Single node cluster vs multi node cluster compared to possible undetected data-loss Elasticsearch	1	1980	July 5, 2017
What is the status of resiliency with the latest ES version? Elasticsearch	2	1065	February 23, 2018
Elasticsearch 2.3 as primary data store? Elasticsearch	3	7815	July 5, 2017
Elasticsearch as a primary database Elasticsearch	38	78730	July 4, 2017
Elasticsearch as a primary store Elasticsearch	1	382	June 18, 2018

Best ways to prevent data lost in elastic

Related topics