X-Pack Security : Role definition query template with 'terms'

Hi,

I am evaluating Elastic 5 beta1 X-Pack document level security.

I have user metadata object which has attribute 'groups'. The groups is a list of group names.

Assume I have one user who has _user.metadata.groups =["a","b"].
I try to write query template to generate below query:
{"template":{"inline":{"terms":{"acl.groups":["a", "b"] }}}}

My question is
How do I pass variable _user.metadata.groups as "acl.groups' field's value in the template?

Another question about X-Pack security limitation.
From https://www.elastic.co/guide/en/x-pack/current/security-limitations.html
it said 'The terms query with terms lookup isn’t supported.' Is this only under 'remote' context?
The 'terms' query is supported for role definition and user search with current cluster?

Thank you in advance for your replies.

Best Regards,

Jenny

Hi Jenny,

This is only for the case where the query is actually trying to lookup terms in the same index or another index (as in it will execute a search for the terms). The terms query is supported with the terms specified.

In order to template the query for document level security, you can use something like the following:

{
  "indices" : [
    {
      "names" : [ "my_index" ],
      "privileges" : [ "read" ],
      "query" : {
        "template" : {
          "inline" : {
            "terms" : { "acl.groups" : "{{#toJson}}_user.metadata.groups{{/toJson}}" }
          }
        }
      }
    }
  ]
}

The information about was pulled from the following documentation:

Hi,

Thank you for your quick response.

I did try {#toJson}... before posting this question. I could not make it worked.

I tried again I still have an error when I do /_search.

Here my role definition
/_xpack/security/role/xxx_user

{
"xxx_user": {
"cluster": ,
"indices": [
{
"names": [
"xxx"
],
"privileges": [
"read"
],
"query": "{"template":{"inline":{"terms":{"acl.groups":"{{#toJson}}_user.metadata.groups{{/toJson}}" }}}}"
}
],
"run_as": ,
"metadata": {}
}
}

Here is the error I got from _search

{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[terms] query does not support [acl.groups]",
"line": 1,
"col": 24
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "xxx",
"node": "fK79u0DDTzurHANmfur60w",
"reason": {
"type": "parsing_exception",
"reason": "[terms] query does not support [acl.groups]",
"line": 1,
"col": 24
}
}
],
"caused_by": {
"type": "parsing_exception",
"reason": "[terms] query does not support [acl.groups]",
"line": 1,
"col": 24
}
},
"status": 400
}

Do you see any anything I need to change? or you need more information?

Thanks!

I think removing the escaped quotes \" around the toJson tag will help. So I'm thinking it would look like:

"query": "{\"template\":{\"inline\":{\"terms\":{\"acl.groups\": {{#toJson}}_user.metadata.groups{{/toJson}} }}}}"

Thank you for your quick reply.
tried this before: {"template":{"inline":{"terms":{"acl.groups":{{#toJson}}_user.metadata.groups{{/toJson}} }}}}
Here is the error:
Unexpected character ('{' (code 123)): was expecting double-quote to start field name

Thanks for bearing with me. If the value is not quoted then it gets picked up as JSON itself. The whole inline value must be a string for toJson to work:

"query": "{\"template\":{\"inline\":\"{\"terms\":{\"acl.groups\": {{#toJson}}_user.metadata.groups{{/toJson}} }}\"}}"

Alternatively, the 2.x way still seems to work:

"query": "{\"template\":{\"inline\":{\"terms\":{\"acl.groups\": [\"{{#_user.metadata.groups}}\", \"{{.}}\", \"{{/_user.metadata.groups}}\" ] }}}}"

From my _xpack/security/role/xxx_user, I have

"query": "{\"template\":{\"inline\":\"{\"terms\":{\"acl.groups\":{{#toJson}}_user.metadata.groups{{/toJson}} }}\"}}"

From _search, I have

 {
        "type": "exception",
        "reason": "com.fasterxml.jackson.core.JsonParseException: Unexpected character ('t' (code 116)): was expecting comma to separate Object entries\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@599eb6b9; line: 1, column: 27]"
      },

It is possible that ES 5 BETA1 X-Pack has an issue with this feature?

I will try 2.x way as well.

The 2.x way seems work for me.
To be sure, I would like to see if this template works as I expect.
Where I can view the query produced by the template?
I could not find it from slow search log, even I put it on TRACE level.

Also like to know if the 'toJson' format will be supported in later release.

Thanks!

I think the issue I missed is needing to escape the quotes at more levels; I did it a bit differently by using json format for query, template, inline, and then a string for the value of inline. I confirmed that the following works on beta1:

$ curl -XPUT -u elastic 'localhost:9200/_xpack/security/role/terms_tojson' -d '{ "indices": [
{
"names": [
"foo"
],
"privileges": [
"read"
],
"query": {
"template": {
"inline": "{\"terms\":{\"acl.groups\": {{#toJson}}_user.metadata.groups{{/toJson}} }}"
}
}
}
]
}'
Enter host password for user 'elastic':
"role":{"created":true}}

$ curl -u elastic 'localhost:9200/_xpack/security/role/terms_tojson'?pretty
Enter host password for user 'elastic':
{
  "terms_tojson" : {
    "cluster" : [ ],
    "indices" : [
      {
        "names" : [
          "foo"
        ],
        "privileges" : [
          "read"
        ],
        "query" : "{\"template\":{\"inline\":\"{\\\"terms\\\":{\\\"acl.groups\\\": {{#toJson}}_user.metadata.groups{{/toJson}} }}\"}}"
      }
    ],
    "run_as" : [ ],
    "metadata" : { }
  }
}

$ curl -XPUT 'localhost:9200/foo/t/1' -d '{ "acl": { "groups": [ "a", "c" ] } }' -u elastic
Enter host password for user 'elastic':
{"_index":"foo","_type":"t","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}

$ curl -XPUT 'localhost:9200/foo/t/2' -d '{ "acl": { "groups": [ "c" ] } }' -u elastic
Enter host password for user 'elastic':
{"_index":"foo","_type":"t","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}

$ curl -u elastic -XPUT 'localhost:9200/_xpack/security/user/tojson' -d '{ "password": "changeme", "roles": [ "terms_tojson" ], "metadata": { "groups": [ "a" ] } }'
Enter host password for user 'elastic':
{"user":{"created":false}}

$ curl -u tojson 'localhost:9200/foo/_search?pretty'
Enter host password for user 'tojson':
{
  "took" : 40,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "foo",
        "_type" : "t",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "acl" : {
            "groups" : [
              "a",
              "c"
            ]
          }
        }
      }
    ]
  }
}

$ curl -u elastic 'localhost:9200/foo/_search?pretty'
Enter host password for user 'elastic':
{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "foo",
        "_type" : "t",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "acl" : {
            "groups" : [
              "c"
            ]
          }
        }
      },
      {
        "_index" : "foo",
        "_type" : "t",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "acl" : {
            "groups" : [
              "a",
              "c"
            ]
          }
        }
      }
    ]
  }
}

I do not believe this is logged anywhere currently.

This time 'toJson' format works for me.
I think all my questions answered.
Thank you for your great support.