Maximum characters limit text fields in elasticsearch

Elastic Stack Elasticsearch
1

Hi Team,

We are using an API to get the data into elasticsearch and the API will give the entire documents in the response body which will be stored into elasticsearch into message or event.original. Could you please let us know if there is any restriction on the maximum length of characters that a text field can store in elasticsearch. We are on elastic 8.17.3 version.

Thank you in advance.

2

I think this documentation largely answers your question: General recommendations | Elasticsearch Guide [8.17] | Elastic

Other fields (like keyword) have further limitations, but for text fields I think it's however much data you can send to ES up to the http limit which is about 100MB or if configured higher we eventually disallow blobs or text fields above 2GB. In practice if you consider for instance Moby Dick I think that's about ~2MB of text. So it's probably safe that anything human-consumable would be well below that threshold.

And generally my guidance architecturally is if you aren't sure how large your documents will be or have much larger machine generated or machine consumed documents that may be larger than say 100MB, I would consider storing them in a key/value store that's typically associated with landing data and then index the parts of it relevant for search or return them either as part of a separate request or at least make multiple requests under the hood to retrieve and combine that data.

Happy to discuss further if you have other questions!

1 Like