# Can elasticsearch handle long text? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 26, 2019, 6:25pm UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991 "2019-03-26T18:25:33Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![flash](https://avatars.discourse-cdn.com/v4/letter/f/f475e1/32.png) [@flash](https://discuss.elastic.co/u/flash)\ **Post date:** [March 26, 2019, 6:25pm UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/1 "2019-03-26T18:25:33Z")
I am trying to use elasticsearch to do some document understanding. Here is the index settings ``` DELETE index PUT index/_doc/type_name { "id": "id1" "name": "Lake Tahoe" "indexed_at": "2019-03-01" } PUT index/_doc/type_name { "id": "id2" "name": "Nevada" "indexed_at": "2019-03-01" } PUT index/_doc/type_name { "id": "id3" "name": "California" "indexed_at": "2019-03-01" } PUT index/_doc/type_name { "id": "id4" "name": "Texas" "indexed_at": "2019-03-01" } ``` I want to extract all of the related terms (e.g. "Lake Tahoe", "California", "Nevada" etc.) from such a long text/document: > "Lake Tahoe is a large freshwater lake in the Sierra Nevada Mountains, straddling the border of California and Nevada. It’s known for its beaches and ski resorts." Is it feasible? How shall I index and which query pattern shall I use? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 26, 2019, 11:05pm UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/2 "2019-03-26T23:05:02Z")
I think you can use the percolator API for something similar to what you described. A different way may be [https://github.com/spinscale/elasticsearch-ingest-opennlp](https://github.com/spinscale/elasticsearch-ingest-opennlp) but may be your example was only theorical and not the real use case... ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [March 27, 2019, 9:25am UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/3 "2019-03-27T09:25:00Z")
> [@flash](#): > > Lake Tahoe is a large freshwater lake in the Sierra Nevada Mountains, straddling the border of California and Nevada. It’s known for its beaches and ski resorts There are commercial vendors out there too e.g. try the demo at [Products | LSEG](http://www.opencalais.com/opencalais-demo/) Whatever tool you pick, the [new annotated\_text field type](https://www.elastic.co/blog/search-for-things-not-strings-with-the-annotated-text-plugin) is specifically designed to index this type of content in elasticsearch. ---
**Author:** ![flash](https://avatars.discourse-cdn.com/v4/letter/f/f475e1/32.png) [@flash](https://discuss.elastic.co/u/flash)\ **Post date:** [March 28, 2019, 12:54am UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/4 "2019-03-28T00:54:37Z")
Thanks for replying, @dadoonet ! I want to handle both short-text as well as long-text/document, so not sure if I changed how it index would help? Or does it mean for every entity that I want to stored, I should add "percolator" type into the index? ---
**Author:** ![flash](https://avatars.discourse-cdn.com/v4/letter/f/f475e1/32.png) [@flash](https://discuss.elastic.co/u/flash)\ **Post date:** [March 28, 2019, 12:57am UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/5 "2019-03-28T00:57:15Z")
Thanks @Mark_Harwood! I have the same question ditto. If I want to retrieve the entities I stored in ElasticSearch for both short text such as "Lake Tahoe resort" as well as the long text I listed as the above example. Does it mean I have to change every single entities that I stored to be with "annotated\_text" field type? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [March 28, 2019, 7:52am UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/6 "2019-03-28T07:52:03Z")
> [@flash](#): > > Does it mean I have to change every single entities that I stored to be with "annotated\_text" field type? No. Entity extraction tools like OpenNLP, Rosette or OpenCalais mine structured data (people, places, companies) from unstructured text. Ordinarily you store the structured data in structured ‘keyword’ type fields and keep the original text as a ‘text’ type field. However, if you choose to use the new ‘annotated\_text’ field type you can upgrade your text to include both free text and the structured keywords discovered by your choice of entity extraction tool. You don’t have to use the annotated text field type but the blog I shared illustrates the benefits of this technique. I wrote the support for annotated\_text out of the frustration in dealing with plain ‘keyword’ and ‘text’ fields. For example, an aggregation on the keyword field would tell me “John F Kennedy” is mentioned in some docs but I could never see _where_ in a long text he was mentioned. The highlighter would not work on the text field because the text mentioned “JFK” - not “John F Kennedy”. All traceability of what the entity extraction tool discovered (JFK = John F Kennedy) was lost. With the annotated text field we can highlight mentions of extracted entities like this because we weave the structured keyword values into the indexed text. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 25, 2019, 7:52am UTC](https://discuss.elastic.co/t/can-elasticsearch-handle-long-text/173991/7 "2019-04-25T07:52:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.