Whole phrases searching in large texts in ElasticSearch take a long of time


(Mateusz Aaa) #1

I store Tomcat's logs in ElasticSearch service. Logs colud have over 5k characters. When i search by only timestamp, searching is finished in miliseconds. When indices count was short, searching was fast, but now count of indices is about 370 000 and sarching by whole phrase take a lot of time. I tried to use filter or post_filter and search phrases in results, but it takes the same period of time. I used keyword tokenizer because I want to search whole phrases with special characters like "/" in logLines. How can I improve my query or mappings to speed up my searching?

This is my mappings:

{
"settings": {
"analysis": {
"analyzer": {
"logs_string_analyzer": {
"type": "custom",
"filter": [
"lowercase"
],
"tokenizer": "keyword"
}
}
}
},
"mappings": {
"log": {
"properties": {
"level": {
"type": "long"
},
"logLines": {
"type": "text",
"analyzer": "logs_string_analyzer"
},
"timestamp": {
"type": "long"
}
}
}
}
}
This is my query:
{
"size" : 30,
"query" : {
"bool" : {
"must" : [
{
"wildcard" : {
"logLines" : {
"wildcard" : "XYZ",
"boost" : 1.0
}
}
},
{
"range" : {
"timestamp" : {
"from" : null,
"to" : 1495705598305,
"include_lower" : true,
"include_upper" : true,
"boost" : 2.0
}
}
},
{
"range" : {
"timestamp" : {
"from" : 1495705501496,
"to" : null,
"include_lower" : false,
"include_upper" : true,
"boost" : 2.0
}
}
}
]
}
},
"sort" : [
{
"timestamp" : {
"order" : "desc"
}
}
]
}


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.