Hello,
I writing a GUI to search over my data (network device syslogs) which is index in daily logs.
The GUI is very simple, and uses a java based server.
There is a time range component to allow the user to select a start and an end date.
based on the parameters entered in the GUI I build a search request.
my problem is with the time range.
from the start and end date, I can list the days in between.
ex:
start = 15th sept 2016
end = 21rst sept 2016
-> list= {15-09-2016, 16-09-2016,..., 21-09-2016}
with this list I can build a list of indexes to search in.
BUT
some index in the list may not exist, for example, here, the user sets the end date to a date in the future.
Right now I use the wildcard "" in the index name and elasticsearch doesn't complain.
but the search performance are bad.
If I remove the "", then the search query fails with error 404 if the index doesn't exist.
I could check whether each index exist but I'm sure that this will impact the performances.
I'm looking for ideas/advices on how to handle this properly.
here is an example of a search query
http://10.30.19.110:9200/ubilogs-mbr*-2016-09-17,ubilogs-mbr*-2016-09-18,ubilogs-mbr*-2016-09-19,ubilogs-mbr*-2016-09-20,ubilogs-mbr*-2016-09-21/_search?sort=date:desc
{
"fields": [
"_id",
"_timestamp",
"_source"
],
"query": {"bool": {
"must": [
{"query_string": {
"default_field": "rawlog",
"query": "*",
"default_operator": "AND"
}},
{"query_string": {
"default_field": "logs.device_id",
"query": "OTO3921 OR OTO4045 OR OTO3958 OR OTO709"
}},
{"query_string": {
"default_field": "logs.customer_id",
"query": "449"
}},
{"range": {"logs.date": {
"from": "2016-09-18 08:01:36",
"to": "2016-09-20 08:01:36"
}}}
],
"should": [],
"must_not": []
}},
"from": 0,
"size": 10,
"highlight": {
"number_of_fragments": 0,
"encoder": "html",
"pre_tags": ["<span style=\"background-color:#FFFF00;font-weight:bold\">"],
"post_tags": ["<\/span>"],
"fields": {"rawlog": {}}
}
}
Antoine