# Elasticsearch 7.7 crashing for Term Query if term text 200 - 300 char
**URL:** https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395
**Category:** Elasticsearch
**Created:** [July 16, 2020, 6:44am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395 "2020-07-16T06:44:21Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@pappukhode](https://discuss.elastic.co/u/pappukhode)
#### Post date: [July 16, 2020, 6:44am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/1 "2020-07-16T06:44:21Z")
Hi,
I have upgraded Elasticsearch **5.5** to **7.7** recently.
I have only **1 index** of **30 fields** and **6K data**. Dataset is very simple in nature. [provided 1g memory in jvm options]
In the dataset i have a field `description` which has text around `250-400 chars`.
I am using a search query having combination of `bool, must, query_string, term`.
When I perform a search operation for exact indexed description having 250 chars, search query takes long time to respond [15 sec] and Elasticsearch gets crashed.
If I use small search term of 20-25 chars ES works well.
All the words in search term are fuzzy term, we have appended ~ at end of each word.
In ES 5.5 above scenario was working pretty well with no issue and looks like something has broken in ES 7.7
1. Could you please suggest how should I proceed on my issue?
2. Is there any limit to input search term ?
3. How much memory should I set in development and production ?
---
### Author:  [@Ignacio\_Vera](https://discuss.elastic.co/u/Ignacio_Vera)
#### Post date: [July 16, 2020, 7:15am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/2 "2020-07-16T07:15:50Z")
Could you share a bit more information about the crash? (e.g stacktrace)
---
### Author:  [@pappukhode](https://discuss.elastic.co/u/pappukhode)
#### Post date: [July 16, 2020, 7:22am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/3 "2020-07-16T07:22:57Z")
Its out of memory. I could see lot of logs like
`[gc][2119] overhead, spent [3.7s] collecting in the last [6s]`
Do you feel Fuzzy is consuming lot of memory?
Above query without fuzzy taking 500ms and with fuzzy taking 11 sec with gc log and gets crashed.
---
### Author:  [@Ignacio\_Vera](https://discuss.elastic.co/u/Ignacio_Vera)
#### Post date: [July 16, 2020, 7:31am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/4 "2020-07-16T07:31:54Z")
I think you are hitting this Lucene issue:
[https://issues.apache.org/jira/browse/LUCENE-9286](https://issues.apache.org/jira/browse/LUCENE-9286)
But it is difficult to tell without a heap dump. Maybe you can try to get the hot threads when running the query?
---
### Author:  [@pappukhode](https://discuss.elastic.co/u/pappukhode)
#### Post date: [July 16, 2020, 8:33am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/5 "2020-07-16T08:33:06Z")
@Ignacio_Vera
[2020-07-16T01:44:40,518][WARN][o.e.m.j.JvmGcMonitorService] [gc][2118] overhead, spent [1.1s] collecting in the last [1.1s]
[2020-07-16T01:44:46,589][WARN][o.e.m.j.JvmGcMonitorService] [gc][2119] overhead, spent [3.7s] collecting in the last [6s]
[2020-07-16T01:44:46,170][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [HJL013760] fatal error in thread [elasticsearchsearch][T#11]], exiting
java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.util.ArrayUtil.growExact(ArrayUtil.java:302) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:311) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.Automaton$Builder.addTransition(Automaton.java:770) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.UTF32ToUTF8.all(UTF32ToUTF8.java:251) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.UTF32ToUTF8.end(UTF32ToUTF8.java:231) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.UTF32ToUTF8.build(UTF32ToUTF8.java:194) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.UTF32ToUTF8.convertOneEdge(UTF32ToUTF8.java:137) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.UTF32ToUTF8.convert(UTF32ToUTF8.java:307) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:237) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.util.automaton.CompiledAutomaton.(CompiledAutomaton.java:140) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.search.FuzzyTermsEnum.buildAutomata(FuzzyTermsEnum.java:154) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.apache.lucene.search.FuzzyQuery.(FuzzyQuery.java:111) ~[lucene-core-8.5.1.jar:8.5.1 edb9fc409398f2c3446883f9f80595c884d245d0 - ivera - 2020-04-08 08:55:42]
at org.elasticsearch.index.mapper.StringFieldType.fuzzyQuery(StringFieldType.java:78) ~[elasticsearch-7.7.0.jar:7.7.0]
at org.elasticsearch.index.search.QueryStringQueryParser.getFuzzyQuerySingle(QueryStringQueryParser.java:466) ~[elasticsearch-7.7.0.jar:7.7.0]
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 16, 2020, 8:51am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/6 "2020-07-16T08:51:55Z")
> [@pappukhode](#):
>
> In the dataset i have a field `description` which has text around `250-400 chars` .
> [@pappukhode](#):
>
> a search operation for exact indexed description
Is this field tokenized? Can you share the mapping?
Generally, looking for similar texts would be done using tokenised fields and using the [more like this](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-mlt-query.html) query.
Searching long untokenized fields with fuzzy will be expensive and only allows for max 2 characters difference between search string and matched values.
---
### Author:  [@pappukhode](https://discuss.elastic.co/u/pappukhode)
#### Post date: [July 16, 2020, 9:06am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/7 "2020-07-16T09:06:46Z")
We have not modified or customizing \_mapping. ES be default creating fields and datatypes for \_mappings.
As per elasticsearch behaviour, it has created type as a _text_ and _keyword_ for field description.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 16, 2020, 9:07am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/8 "2020-07-16T09:07:38Z")
> [@pappukhode](#):
>
> , it has created type as a text and keyword for field description.
Which of these 2 fields are you searching?
Seeing the mapping and query would help.
---
### Author:  [@pappukhode](https://discuss.elastic.co/u/pappukhode)
#### Post date: [July 16, 2020, 9:13am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/9 "2020-07-16T09:13:33Z")
**snippet of my query:**
"query": {
"bool": {
"must": [
{
"query\_string": {
"query": "Carl~ Rogers~ founder~ humanistic~ psychology~ movement~ revolutionized~ psychotherapy~ influence~ has~ become~ mainstream~ psychology~ and so on.",
"default\_operator": "AND"
}
},
{
"term": {
"containerName.keyword": "Book Catalog"
}
.....
}
**\_mappings** : description fields looks like below:
`{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}`
If i remove fuzzy ~ char from search terms query works fine. but with ~ char i am facing performance issue and es gets crashed.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [July 16, 2020, 9:29am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/10 "2020-07-16T09:29:26Z")
OK. That's searching the tokenized field but using fuzzy on everything which is expensive.
If you do a lot of this type of fuzzy matching it's probably more efficient to use [ngrams](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-ngram-tokenizer.html)
e.g.
```
PUT my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "my_tokenizer",
"filter": [
"apostrophe",
"lowercase"
]
}
},
"tokenizer": {
"my_tokenizer": {
"type": "ngram",
"min_gram": 3,
"max_gram": 3,
"output_unigrams": true
}
}
}
},
"mappings": {
"properties": {
"description": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
},
"ngram": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
}
}
POST my_index/_doc/1
{
"description":"Carl Rogers founder humanistic psychology movement revolutionized psychotherapy influence has become mainstream psychology "
}
POST my_index/_doc/_search
{
"query": {
"query_string": {
"query": "Carl Rogers founder humanistic sychology movement revolutionised psychotherapy influence has become mainstream",
"default_operator": "OR",
"default_field": "description.ngram"
}
}
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 13, 2020, 9:29am UTC](https://discuss.elastic.co/t/elasticsearch-7-7-crashing-for-term-query-if-term-text-200-300-char/241395/11 "2020-08-13T09:29:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.