Need advise: time range searcha and how to search over multiple daily indexes

Elastic Stack Elasticsearch
1

Hello,

I writing a GUI to search over my data (network device syslogs) which is index in daily logs.
The GUI is very simple, and uses a java based server.
There is a time range component to allow the user to select a start and an end date.

based on the parameters entered in the GUI I build a search request.
my problem is with the time range.
from the start and end date, I can list the days in between.
ex:
start = 15th sept 2016
end = 21rst sept 2016
-> list= {15-09-2016, 16-09-2016,..., 21-09-2016}
with this list I can build a list of indexes to search in.
BUT
some index in the list may not exist, for example, here, the user sets the end date to a date in the future.

Right now I use the wildcard "" in the index name and elasticsearch doesn't complain.
but the search performance are bad.
If I remove the "", then the search query fails with error 404 if the index doesn't exist.

I could check whether each index exist but I'm sure that this will impact the performances.

I'm looking for ideas/advices on how to handle this properly.

here is an example of a search query

http://10.30.19.110:9200/ubilogs-mbr*-2016-09-17,ubilogs-mbr*-2016-09-18,ubilogs-mbr*-2016-09-19,ubilogs-mbr*-2016-09-20,ubilogs-mbr*-2016-09-21/_search?sort=date:desc
 {
    "fields":     [
        "_id",
        "_timestamp",
        "_source"
    ],
    "query": {"bool":     {
        "must":         [
            {"query_string":             {
                "default_field": "rawlog",
                "query": "*",
                "default_operator": "AND"
            }},
            {"query_string":             {
                "default_field": "logs.device_id",
                "query": "OTO3921 OR OTO4045 OR OTO3958 OR OTO709"
            }},
            {"query_string":             {
                "default_field": "logs.customer_id",
                "query": "449"
            }},
            {"range": {"logs.date":             {
                "from": "2016-09-18 08:01:36",
                "to": "2016-09-20 08:01:36"
            }}}
        ],
        "should": [],
        "must_not": []
    }},
    "from": 0,
    "size": 10,
    "highlight":     {
        "number_of_fragments": 0,
        "encoder": "html",
        "pre_tags": ["<span style=\"background-color:#FFFF00;font-weight:bold\">"],
        "post_tags": ["<\/span>"],
        "fields": {"rawlog": {}}
    }
}

Antoine

2

Maybe the ignore_unavailable option documented here

https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-index.html

would help you?

Isabel

3

this is exactly what I needed, thank you very much

Antoine