# How Can I Identify LARGE Documents/Logs

**URL:** <https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313>\
**Category:** Elasticsearch\
**Created:** [April 13, 2016, 9:50pm UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313 "2016-04-13T21:50:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Maxwell\_Flanders](https://avatars.discourse-cdn.com/v4/letter/m/eb8c5e/32.png) [@Maxwell\_Flanders](https://discuss.elastic.co/u/Maxwell_Flanders)\
**Post date:** [April 13, 2016, 9:50pm UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/1 "2016-04-13T21:50:17Z")

</div>

We have a 2-master, 5-slave elasticsearch cluster collecting logs from a ton of different microservice servers. Although indexing has never been a problem, occasionally, our kibana goes down due extremely long timeouts. Sometimes I have been able to track these problems back to EXTREMELY large individual documents ruining query times. Typically these have been the result of a faulty multiline filter.

However here is my problem - sometimes when we get these time-out issues, I don't know how to identify what server is producing the massive logs because we have so many. Since most of our logs go into the same, daily index, is there any way to identify based on source (we have a "source" field in our logs) or something else which server is producing the problem logs that are freezing our queries??

Any help on this would be massively appreciated!!  
Thanks!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 13, 2016, 11:22pm UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/2 "2016-04-13T23:22:34Z")

</div>

You might need to install the `_size` plugin to help grab the document size - [https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-size.html](https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-size.html)

Also, there is no such thing as a slave in ES 🙂

---

<div class="post-metadata">

**Author:** ![Maxwell\_Flanders](https://avatars.discourse-cdn.com/v4/letter/m/eb8c5e/32.png) [@Maxwell\_Flanders](https://discuss.elastic.co/u/Maxwell_Flanders)\
**Post date:** [April 14, 2016, 1:30am UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/3 "2016-04-14T01:30:07Z")

</div>

That's a fantastic plugin, thank you. Hopefully I can put it to good use. Any other ideas are still appreciated in the meantime!

Am I calling the non-master nodes incorrectly??

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 14, 2016, 3:54am UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/4 "2016-04-14T03:54:46Z")

</div>

Yeah, there is a single active master and multiple master eligable.

---

<div class="post-metadata">

**Author:** ![Maxwell\_Flanders](https://avatars.discourse-cdn.com/v4/letter/m/eb8c5e/32.png) [@Maxwell\_Flanders](https://discuss.elastic.co/u/Maxwell_Flanders)\
**Post date:** [April 14, 2016, 2:53pm UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/5 "2016-04-14T14:53:24Z")

</div>

Oh, I had been referring to 5 data-only, non-master eligible nodes as slaves. We have 1 master and 1 master eligible.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 14, 2016, 11:01pm UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/6 "2016-04-14T23:01:23Z")

</div>

> [@Maxwell\_Flanders](#):
>
> We have 1 master and 1 master eligible.

That's not good. Read [Important Configuration Changes | Elasticsearch: The Definitive Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/master/important-configuration-changes.html#_minimum_master_nodes)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:59pm UTC](https://discuss.elastic.co/t/how-can-i-identify-large-documents-logs/47313/7 "2017-07-05T22:59:03Z")

</div>


