For a while i've been using Elasticsearch and I have a few questions regarding reliability, replication and ingest processors.
I read the following page:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-replication.html
For this example we have the following setup:
3 Elasticsearch nodes in a cluster (master + data + ingest on every node)
1 Logstash
When you send a document to Elasticsearch, with Logstash, it will initially arrive in memory.
- What happens if in this exact moment the node, that receives the document, will die?
- Even if the connection is TCP is this also on the application level, does this mean that logstash will re-send the message or can it happen that a document will get lost?
- How are ingest processors implemented, before or after replication?
- When an ingest node dies could there be a possibility that a received document will be lost, or does Logstash just resend the document?