Filtered alias and possible options for forcing filters for queries

Elastic Stack Elasticsearch
1

In our application we have the need to restrict access to data. We would
like to use filters for that. The filtered aliashttp://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html seems
like a nice way to do that and my original thought was to simply create an
alias for each user and then add a filter to it as needed.

That would work quite nicely, however we split the content potentially
across many indices (think logstash and it's default per day index).

Sadly a filter alias can only point to one index (wildcards don't seem to
work in the 'index' either), which I assume is due to making it also work
for document posting and updates ....

With that said, is there something obvious I'm missing to get the desired
functionality of applying/forcing a filter dependent on a user?

Right now I'm considering 2 choices:

  1. create per user aliases for each index they should have access to and
    then running the search against /*/. That seems like it quickly
    becomes hard to scale and manage

  2. store the user filters in ES and having a shim that inserts them at
    query time. Right now I already use a shim to authenticate via an HTTP
    header token through an nginx proxy

Any thoughts and opinions would be most welcome.

@matthias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9001455a-f75a-4161-b475-9d2d266369cc%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi,
I would go for option 1, it just means that while creating the alias, you
need to provide a list of indices, and for each index you 'll need to
provide a filter, which in your case will always be the same if I got it
right.

Does it make sense?

On Friday, January 31, 2014 7:07:02 PM UTC+1, Matthias Johnson wrote:

In our application we have the need to restrict access to data. We would
like to use filters for that. The filtered aliashttp://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html seems
like a nice way to do that and my original thought was to simply create an
alias for each user and then add a filter to it as needed.

That would work quite nicely, however we split the content potentially
across many indices (think logstash and it's default per day index).

Sadly a filter alias can only point to one index (wildcards don't seem to
work in the 'index' either), which I assume is due to making it also work
for document posting and updates ....

With that said, is there something obvious I'm missing to get the desired
functionality of applying/forcing a filter dependent on a user?

Right now I'm considering 2 choices:

  1. create per user aliases for each index they should have access to and
    then running the search against /*/. That seems like it quickly
    becomes hard to scale and manage

  2. store the user filters in ES and having a shim that inserts them at
    query time. Right now I already use a shim to authenticate via an HTTP
    header token through an nginx proxy

Any thoughts and opinions would be most welcome.

@matthias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/93f636e7-882e-4261-92e1-fc13e8d0ff5c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

It does, and that would leave me with only needing to solve the issue of a
document update/load, since I won't be able to index against the alias with
multiple indexes.

For example:

{
"actions" : [
{ "add" : { "index" : "doc_1", "alias" : "test", "filter" : { "term" : {
"_all" : "world" } } } },
{ "add" : { "index" : "doc_2", "alias" : "test", "filter" : { "term" : {
"_all" : "world" } } } }
.....
]
}

If I end up creating a new index, I will need to update all user's aliases
to add the new index. Although if it is missed, the worst case scenario
will be "denied" access, rather than accidental leaking of data. That feels
like a decent default.

And for those with a filter I'll have to add it to each index line. It
would be nice if the filter could be specified also globally for the alias.

That will require some careful syncing, but should be reasonable.

I will also need to solve the issue of possible updates to the alias and
filter. I suspect that my auth layer can likely validate that the doc is
accessibly via a HEAD request to the alias and if it shows up as visible I
can permit the update to the actual index instead of the alias ...

@matthias

On Monday, February 3, 2014 3:47:12 AM UTC-7, Luca Cavanna wrote:

Hi,
I would go for option 1, it just means that while creating the alias, you
need to provide a list of indices, and for each index you 'll need to
provide a filter, which in your case will always be the same if I got it
right.

Does it make sense?

On Friday, January 31, 2014 7:07:02 PM UTC+1, Matthias Johnson wrote:

In our application we have the need to restrict access to data. We would
like to use filters for that. The filtered aliashttp://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html seems
like a nice way to do that and my original thought was to simply create an
alias for each user and then add a filter to it as needed.

That would work quite nicely, however we split the content potentially
across many indices (think logstash and it's default per day index).

Sadly a filter alias can only point to one index (wildcards don't seem to
work in the 'index' either), which I assume is due to making it also work
for document posting and updates ....

With that said, is there something obvious I'm missing to get the desired
functionality of applying/forcing a filter dependent on a user?

Right now I'm considering 2 choices:

  1. create per user aliases for each index they should have access to and
    then running the search against /*/. That seems like it quickly
    becomes hard to scale and manage

  2. store the user filters in ES and having a shim that inserts them at
    query time. Right now I already use a shim to authenticate via an HTTP
    header token through an nginx proxy

Any thoughts and opinions would be most welcome.

@matthias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/64454d01-f454-4ac3-8eff-9d1e984ea1b6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.