# Does Elastic Search store all log data sent from File Beat?

**URL:** <https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214>\
**Category:** Elasticsearch\
**Created:** [March 12, 2016, 12:05am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214 "2016-03-12T00:05:00Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sameer\_Panicker](https://avatars.discourse-cdn.com/v4/letter/s/d9b06d/32.png) [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\
**Post date:** [March 12, 2016, 12:05am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/1 "2016-03-12T00:05:00Z")

</div>

I have configured a log path inside FB configuration which has data around 10GB. This is my server 1.

I have ES installed on my Server 2. FB is configured to send data to ES from Server 1 to Server 2.

ES is used to store data, will it actually pull/copy all 10GB of data onto my Server 2 OR it wont physically copy/pull 10GB of files from Server 1, rather it will just use those log files present on Server 1 to filter.

How does it actually works?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 12, 2016, 12:14am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/2 "2016-03-12T00:14:08Z")

</div>

ES will store anything that FB sends to it.  
So as long as you aren't filtering on the FB end, then it'll be everything you point it at.

---

<div class="post-metadata">

**Author:** ![Sameer\_Panicker](https://avatars.discourse-cdn.com/v4/letter/s/d9b06d/32.png) [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\
**Post date:** [March 12, 2016, 12:41am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/3 "2016-03-12T00:41:50Z")

</div>

By default what filter value is set on FB. Where can i see those filter? Bcoz i have not configured any specific filter from my end.

Also, i see the same disk size on my server 2 and all my logs are also getting searched. How is that occurring ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 12, 2016, 12:52am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/4 "2016-03-12T00:52:38Z")

</div>

> [@Sameer\_Panicker](#):
>
> By default what filter value is set on FB. Where can i see those filter? Bcoz i have not configured any specific filter from my end.

None.

> [@Sameer\_Panicker](#):
>
> Also, i see the same disk size on my server 2 and all my logs are also getting searched. How is that occurring ?

Can you elaborate on this a bit more, I am not following what you are asking.

---

<div class="post-metadata">

**Author:** ![Sameer\_Panicker](https://avatars.discourse-cdn.com/v4/letter/s/d9b06d/32.png) [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\
**Post date:** [March 12, 2016, 1:04am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/5 "2016-03-12T01:04:24Z")

</div>

The server disk space should go low when fb sends all the 10gb of data to my elastic search server. Currently, i dont see any degradation in my servers disk space.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 12, 2016, 1:07am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/6 "2016-03-12T01:07:20Z")

</div>

Filebeat does not delete the files once it has read them, that is up to you to manage.

---

<div class="post-metadata">

**Author:** ![Sameer\_Panicker](https://avatars.discourse-cdn.com/v4/letter/s/d9b06d/32.png) [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\
**Post date:** [March 12, 2016, 1:26am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/7 "2016-03-12T01:26:25Z")

</div>

May be i m missing a configuration here. Can u tell me where can i see all the stored by ES ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 12, 2016, 1:32am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/8 "2016-03-12T01:32:20Z")

</div>

Are you using Kibana? That's the most visual way.  
Otherwise maybe something like `_cat/indices`.

---

<div class="post-metadata">

**Author:** ![Sameer\_Panicker](https://avatars.discourse-cdn.com/v4/letter/s/d9b06d/32.png) [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\
**Post date:** [March 12, 2016, 2:25am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/9 "2016-03-12T02:25:25Z")

</div>

Yes i m using kibana.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 12, 2016, 2:36am UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/10 "2016-03-12T02:36:42Z")

</div>

Using the Discover page in Kibana will show you all the data.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:08pm UTC](https://discuss.elastic.co/t/does-elastic-search-store-all-log-data-sent-from-file-beat/44214/11 "2017-07-05T23:08:55Z")

</div>


