# ES flow

**URL:** <https://discuss.elastic.co/t/es-flow/188220>\
**Category:** Elasticsearch\
**Created:** [July 1, 2019, 3:41am UTC](https://discuss.elastic.co/t/es-flow/188220 "2019-07-01T03:41:21Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![suvarna](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@suvarna](https://discuss.elastic.co/u/suvarna)\
**Post date:** [July 1, 2019, 3:41am UTC](https://discuss.elastic.co/t/es-flow/188220/1 "2019-07-01T03:41:21Z")

</div>

Hi ,

Can you please explain the data and control flow of index write in Elasticsearch.

Like when we send a documents, how the documents flow from DRAM to persistence device(SSD) .

Thanks

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [July 1, 2019, 6:55am UTC](https://discuss.elastic.co/t/es-flow/188220/2 "2019-07-01T06:55:02Z")

</div>

It's a bit difficult to know how to answer such a broad question. Can you be more specific? What research have you already done into this question?

---

<div class="post-metadata">

**Author:** ![suvarna](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@suvarna](https://discuss.elastic.co/u/suvarna)\
**Post date:** [July 2, 2019, 3:46am UTC](https://discuss.elastic.co/t/es-flow/188220/3 "2019-07-02T03:46:47Z")

</div>

Hi

I need the data flow for example:

> Initially New documents are collected in an in-memory indexing buffer .  
> Every so often, the buffer is commited:  
> A new segments a supplementary inverted index is written to disk.  
> A new commit point is written to disk, which includes the name of the new segment.  
> The disk is fsync’ed all writes waiting in the filesystem cache are flushed to disk, .  
> The new segment is opened, making the documents it contains visible to search.  
> The in-memory buffer is cleared, and is ready to accept new documents.

Like wise i need indexing process steps that how Elasticsearch uses heap and off heap memory.

what is the trigger point for flush. and from where flush happens is it from buffer or from translogs .

It will be great if you provide the data flow from heap to disk.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 2, 2019, 5:18am UTC](https://discuss.elastic.co/t/es-flow/188220/4 "2019-07-02T05:18:34Z")

</div>

What do you need this for? What problem are you trying to solve?

[This chapter](https://www.elastic.co/guide/en/elasticsearch/guide/current/inside-a-shard.html) describes how a shard works and is probably still largely valid even if it is a bit old.

---

<div class="post-metadata">

**Author:** ![suvarna](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@suvarna](https://discuss.elastic.co/u/suvarna)\
**Post date:** [July 5, 2019, 3:53am UTC](https://discuss.elastic.co/t/es-flow/188220/5 "2019-07-05T03:53:58Z")

</div>

Thanks Christian .

I got it what i was looking for .

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2019, 3:58am UTC](https://discuss.elastic.co/t/es-flow/188220/6 "2019-08-02T03:58:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.