# Is the document a file that is ‘closed’?

**URL:** <https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757>\
**Category:** Elasticsearch\
**Created:** [October 24, 2016, 1:45pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757 "2016-10-24T13:45:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![akumar11](https://avatars.discourse-cdn.com/v4/letter/a/b487fb/32.png) [@akumar11](https://discuss.elastic.co/u/akumar11)\
**Post date:** [October 24, 2016, 1:45pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/1 "2016-10-24T13:45:54Z")

</div>

Hello All,

First post here and hopefully an easy question. I was asked: if Elastic is accepting streaming data, does that file have to be closed before it can be fed into Lucene to be indexed?

I'm in my second week researching Elasticsearch and this question doesn't entirely make sense. Here's my answer (quoted from the documentation):

Accepting streaming data would the job of the Bulk API. If the document is changed and saved the timestamp/date would be differ, therefore the document will be replaced with the latest version and a higher version number. The latest version will be indexed when the Update action is called.

When using update action \_retry\_on\_conflict can be used as field in the action itself (not in the extra payload line), to specify how many times an update should be retried in the case of a version conflict.  
The update action payload, supports the following options: doc (partial document), upsert,doc\_as\_upsert, script, params (for script), lang (for script) and fields.

I would like to know if there is a better answer to the question.

---

<div class="post-metadata">

**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\
**Post date:** [October 25, 2016, 12:20pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/2 "2016-10-25T12:20:33Z")

</div>

I think a first step to a better answer would be to gather more data on what the actual use case is behind "if Elastic is accepting streaming data...".

If we are talking about log data to be streamed to Elasticsearch, you might want to look at [https://www.elastic.co/products/beats/filebeat](https://www.elastic.co/products/beats/filebeat) or [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html) on ways to stream lines from logfiles to Elasticsearch as they come in.

(Also, as a general hint: If a question you get doesn't entirely make sense, don't be afraid to ask for clarifications until it makes sense. Often there's small things missing that are obvious to the one asking the question but way less obvious to the one trying to come up with an answer.)

Hope this helps,  
Isabel

---

<div class="post-metadata">

**Author:** ![akumar11](https://avatars.discourse-cdn.com/v4/letter/a/b487fb/32.png) [@akumar11](https://discuss.elastic.co/u/akumar11)\
**Post date:** [October 25, 2016, 1:22pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/3 "2016-10-25T13:22:55Z")

</div>

That's great information and advice. It helped me direct my searches better.

Thanks Isabel.

---

<div class="post-metadata">

**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\
**Post date:** [October 25, 2016, 1:58pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/4 "2016-10-25T13:58:56Z")

</div>

Glad I could help. Good luck with your Elasticsearch project. Would love to hear more about what you are actually doing with the project if you care to share.

Isabel

---

<div class="post-metadata">

**Author:** ![akumar11](https://avatars.discourse-cdn.com/v4/letter/a/b487fb/32.png) [@akumar11](https://discuss.elastic.co/u/akumar11)\
**Post date:** [October 25, 2016, 2:29pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/5 "2016-10-25T14:29:53Z")

</div>

I'm not doing anything specific yet, just researching and learning as much as I can. I do have a lot of log data that would be nice to visualize.

---

<div class="post-metadata">

**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\
**Post date:** [October 25, 2016, 4:36pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/6 "2016-10-25T16:36:01Z")

</div>

If you end up doing the visualization with Elasticsearch (and friends like Beats, Logstash, Kibana) let us know 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:09pm UTC](https://discuss.elastic.co/t/is-the-document-a-file-that-is-closed/63757/7 "2017-07-05T22:09:56Z")

</div>


