A clarification about index aliases is needed here

Elastic Stack Elasticsearch
1

Hi,

Is it possible to create an index alias that when users will connect to (for example by using Kibana) they will see only part of the data (for example - only SOME of the types that exist on the relevant index)?

I tried to create an alias like this:
curl -XPOST 'http://mvs-es04:9200/_aliases' -d '{
"actions": [
{
"add": {
"index": "logstash-2016.05.16",
"alias": "crm-2016.05.16",
"filter": {
"match": {
"_type": "syslog_f5"
}
}
}
}
]
}'

And when I entered the "Discover" tab on my Kibana and selected the new index pattern (crm-*) I still could see all the data on the index. Is that normal?

Thanks,
Yuval.

2

You can create an alias with a filter - https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-aliases.html#filtered

3

Hi,

Thanks warkolm... but - isn't that exactly what I did? I tried that in the Discover tab in Kibana but got results that didn't match the filter... Maybe I'm doing something wrong here? Can you please provide an example of how to create the filtered alias and how to test it?

Sorry for the newbie question...
Thanks,
Yuval.

4

https://www.elastic.co/guide/en/elasticsearch/guide/current/faking-it.html

5

Hi again,

Thanks a lot. It seems that it works but... one more question: in the example you provided they used a term filter, I tried that now and it works just fine but is it possible to have a filter like fieldA:xxx AND (fieldB:yyy OR fieldC:zzz OR fieldD:"xxxyyyzzz") AND NOT fieldE:("AAA" "BBB" "CCC")

As you can see above I tried to use a match filter (at least that what I thought that I was doing...) and it didn't work - the alias seemed to contain all the data.

Is that possible? if so, how...

Thanks - again...
Yuval.

6

Hi again,

I just tried this command to generate the alias:

curl -XPOST 'http://es:9200/_aliases' -d '{
    "actions" : [
        {
            "add" : {
                 "index" : "logstash-2016.05.18",
                 "alias" : "filtered1-2016.05.18",
                 "filter" : { "query_string" : { "query" : "NOT ls_host.raw:mvs-ls" } }
            }
        }
    ]
}'

Here's what I get in my Kibana when I use that alias (look at the left) with a filter that should (to my understanding) return empty result set (since that the filter I used in Kibana is exactly the opposite of the filter that I used in the alias):

Screenshot from 2016-05-18 13:32:36.png1902×666 71.5 KB

Is there something I'm missing?

Tnx,
Yuval.

7

I just wanted to add that now I tried to do the same in cURL (at least I thought it would be same):

curl -XGET 'http://es:9200/filtered1-*/_search?pretty' -d '
> {
>    "query" : {
>      "query_string": {
>          "query": "ls_host.raw:mvs-ls"
>      }
>    },
>    "sort": [
>      {
>        "@timestamp": {
>          "order": "desc"
>        }
>      }
>    ]
> }'
{
  "took" : 2879,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  }
}

As you can see, the query returned no hits as expected. Is there an explanation for the fact that Kibana returned results from the index directly as if it was bypassing the alias?

8

Just another update: I just tried to use a specific alias in Kibana so instead of specifying the index like 'filtered-*' I specified it with 'filtered-2016.05.18' and it worked just fine. Is it something specific about wildcards?

9

And another update: If I create an index pattern named 'filtered-*' and I check the (not recommended) box labeled "Do not expand index pattern when searching (Not recommended)" everything works as expected. Is that really the best way to do it? in that case - is it recommended or not?