Associating LDAP Roles to Tags

[djtecha]

I'm trying to use the list of Roles associated to an ldap user as a DLS security option. Currently we mark logs with tags upstream and I'm trying to use the template that's in your docs. I have gotten certain things to match such as _user.username or _user.metadata.cn when I passed the data through, but believe this has something to do with Roles being an array.

roles.yml

engineer_user:
  cluster: [ ]
  indices:
    - names: [ 'engineer-*' ]
      privileges: [ 'read' ]
      query: '{
                "template": {
                  "source": {
                   "term":  { "tags.keyword": "{{_user.roles}}" }
                 }
                }
              }'

And here is my associated user:

{
  "username" : "test",
  "roles" : [
    "kibana_user",
    "engineer_user",
    "Engineering-SysOps",
    "Engineering-jenkins-ci-users"
  ],
  "full_name" : null,
  "email" : null,
  "metadata" : {
    "ldap_dn" : "uid=test,ou=Users",
    "ldap_groups" : [
      "cn=Engineering-misc-vpn,ou=Users
    ]
  },
  "enabled" : true
}

And finally a topic that should return:

  {
    "_index": "engineer-2017.11.10",
    "_type": "doc",
    "_id": "AV-nWAXqJXNbdGXr7alm",
    "_score": 2.4451666,
    "_source": {
      "input_type": "log",
      "pid": "32539",
      "source": "/var/log/test.log",
      "message": "REDACTED",
      "type": "log",
      "tags": [ "Engineering-SysOps" ],
      "hostname": "ip-10-10-1-232",
      "@timestamp": "2017-11-10T19:11:22.577Z",
      "beat": {
        "hostname": "ip-10-10-1-232"
      },
      "Level": "DEBUG",
      "time_of_event": "2017-11-10T19:11:22.407085+00:00"
    }

[djtecha]

Kind of feels like {{_user.roles}} is being passed as a sting, or I'm trying to compare 2 arrays (the tags.keyword array and the _users.roles array). Is there a good way to do this comparison here to find if ANY elements match?

[djtecha]

So with templating, shouldn't this be creating 4 queries here? one for each role? So the expanded role would be more like this?

engineer_user:
  cluster: [ ]
  indices:
- names: [ 'engineer-*' ]
  privileges: [ 'read' ]
  query: '{
            "template": {
              "source": {
               "term":  { "tags.keyword": "kibana_user" }
             }
            }
          }'
  query: '{
            "template": {
              "source": {
               "term":  { "tags.keyword": "engineer_user" }
             }
            }
          }'
  query: '{
            "template": {
              "source": {
               "term":  { "tags.keyword": "Engineering-SysOps" }
             }
            }
          }'
  query: '{
            "template": {
              "source": {
               "term":  { "tags.keyword": "Engineering-jenkins-ci-users" }
             }
            }
          }'

[djtecha]

Ok, figured it out. It might help to add an example of a query that uses roles btw in your docs. For others that get stuck this is how I got it working, using the match vs term part of the query:

engineer_user:
  cluster: [ ]
  indices:
- names: [ 'engineering-*' ]
  privileges: [ 'read' ]
  query: '{
            "template": {
              "source": {
               "match":  { "tags": "{{_user.roles}}" }
             }
            }
          }'

[Mike.Barretta]

Actually a bit surprised if this is working as you intend. As far as I know, {{_user.roles}} is being expanded as [0=<tag1>, 1=<tag2>, ...]. To get a nice list, you can use features of the search template semantics, giving you the ability to do

{{#toJson}}_user.roles{{/toJson}}

That should result in your lists rendering as you want.

Your match query is currently masking this difference because you're doing a full text search instead of a keyword match.

However! The way the term/s query works, you essentially are doing an OR: if any _user.roles exist in tags, you'll get a match. Is that the behavior you seek? Or rather, are you looking to say "this document can only be viewed by users with all of the tagged roles? If the former, good! If the latter, things are more complicated. Thankfully, a change is coming (hopefully) in release 6.1 which exposes a new Lucene query allowing the AND logic as well as more complete control of the number/percentage of terms that must match for the whole document to match. With that, you'll be doing a terms_set query in the role template vs a plain terms.

[djtecha]

Do you have a working role you could provide for me here? And you are correct It does match part of the values but turns out it's also matching on things it shouldn't be. I'm trying to just say if there's any tag that exists in the users role array then they should have access to the document.

[Mike.Barretta]

Using your role definition, you should be fine doing:

engineer_user:
  cluster: [ ]
  indices:
- names: [ 'engineering-*' ]
  privileges: [ 'read' ]
  query: '{
     "template": {
         "source": {\"constant_score\": {\"filter\": {\"terms\": {\"tags\": {{#toJson}}_user.roles{{/toJson}} } } } }
     }
  }'

Couple things to note:

• The query is "embedded" under source due to the fact that your _user.roles is a list (mustache template will render it like ["role1", "role2"]) and lists can't be surrounded by quotes. Unescaped quotes mean quotes around your list: "["role1","role2"]"
• I used a constant score filter mostly for performance reasons. I'm guessing your role match logic is not meant to contribute to result scoring (i.e. it's a filter), so constant_score forces the terms match to happen in a filter context; much faster!

[djtecha]

ahh thank you. I was so close Also, I had to change tags to tags.keyword (but that's just a note for me in the future).