# Data stream = real time search?

**URL:** <https://discuss.elastic.co/t/data-stream-real-time-search/263697>\
**Category:** Elasticsearch\
**Created:** [February 9, 2021, 8:57am UTC](https://discuss.elastic.co/t/data-stream-real-time-search/263697 "2021-02-09T08:57:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [February 9, 2021, 8:57am UTC](https://discuss.elastic.co/t/data-stream-real-time-search/263697/1 "2021-02-09T08:57:42Z")

</div>

I dont understand what feature Data Stream ([Data streams | Elasticsearch Reference [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/data-streams.html)) is providing.

On first reading, DataStream is dynamic index alias.

Is it providing in-memory segments for real-time indexing/search?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [February 9, 2021, 10:11am UTC](https://discuss.elastic.co/t/data-stream-real-time-search/263697/2 "2021-02-09T10:11:49Z")

</div>

It's a formalisation of a strategy for dealing with endless firehoses of information.  
New data is likely to be more interesting and is held on fast servers that build indices on disk _and_ serve queries. Older indices are moved off onto less beefy servers that just serve queries Even older data is held in backup storage and queried less frequently.

All these policies govern data ageing and the automatic movement between different classes of data store.

> [@ebuildy](#):
>
> Is it providing in-memory segments for real-time indexing/search?

No, unless you consider that giving a beefy server lots of RAM will provide plenty of file system cache for holding those immutable segment files Lucene creates in memory.

---

<div class="post-metadata">

**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\
**Post date:** [February 9, 2021, 10:36am UTC](https://discuss.elastic.co/t/data-stream-real-time-search/263697/3 "2021-02-09T10:36:29Z")

</div>

Ok got it! thanks you.

This is more a "time series data governance" feature than a "data stream" (in analogy with Kafka/Spark stream).

I know Vespa use in-memory storage to provide super-fast indexing, would be great to have same thing with elasticsearch.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 9, 2021, 10:37am UTC](https://discuss.elastic.co/t/data-stream-real-time-search/263697/4 "2021-03-09T10:37:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.