ES 7.x : Setting threshold for document size and collecting metrics exceeding the same

Elastic Stack Elasticsearch
1

Hi,

Is there any way of setting a global parameter across indices in ES to the effect of defining the limit of size of each document and if the document size in any index exceeding this limit (either as result of indexing or as result of updates to the same document) updates a specific metric of ES ?

Context of usecase : I have a usecase where documents are either indexed immutably or same document updated at higher frequency exposed as a API on top of ES client API. As we know indexing large documents (~100 to ~120kb size per document) and in addition, updating the same is not a good practice. Hence, I wanted to find a way of defining a document-size threshold and track excess of the defined threshold.

Searched around for the configuration and metrics to above effect at ES level, but. could not find any direct way.

Please let me know if I am missing any existing metrics with above capability

  • Dinesh
2

Hi,

Have you tried using the _size field? If not, have a look here.
You could enable the field for your index and then either query all documents with a size above your treshold or create a watcher for this.

Best regards
Wolfram

3

Thanks Wolfram for the idea of creating a watcher so that I can tie it up with a custom alert. Will try that out

  • Dinesh
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.