First post here and hopefully an easy question. I was asked: if Elastic is accepting streaming data, does that file have to be closed before it can be fed into Lucene to be indexed?
I'm in my second week researching Elasticsearch and this question doesn't entirely make sense. Here's my answer (quoted from the documentation):
Accepting streaming data would the job of the Bulk API. If the document is changed and saved the timestamp/date would be differ, therefore the document will be replaced with the latest version and a higher version number. The latest version will be indexed when the Update action is called.
When using update action _retry_on_conflict can be used as field in the action itself (not in the extra payload line), to specify how many times an update should be retried in the case of a version conflict.
The update action payload, supports the following options: doc (partial document), upsert,doc_as_upsert, script, params (for script), lang (for script) and fields.
I would like to know if there is a better answer to the question.