Fields aggregation or field formatter?

(Chris Clifton) #1

Hi, wondering how to combine a text search field for easier searching through our cloudtrail logs in our ELK stack (5.3.0),
We'd like to have a search field called IAM-User = userIdentity.userName or userIdentity.sessionContext.sessionIssuer.userName or requestParameters.roleSessionName

Depending on which one has a value (exists), then search that field.

We're currently doing this : type:cloudtrail AND (userIdentity.userName:"app-prod" OR userIdentity.sessionContext.sessionIssuer.userName:"app-prod" OR requestParameters.roleSessionName:"app-prod")

But this gets really tedious for searching through all our other users.

(Lee Drengenberg) #2

Hi Chris,

I think a scripted field might help. Here's an experiment I did.

  1. I put this data using Kibana dev console. Each doc has a name but in a different field;
PUT /discuss/test/1
    "date" : "2017-05-26T00:01:00",
    "name" : "Lee"

PUT /discuss/test/2
    "date" : "2017-05-26T00:02:00",
    "firstName" : "Bob"

PUT /discuss/test/3
    "date" : "2017-05-26T00:02:30",
    "fullName" : "Bart Simpson"
  1. It looks like this in Discover;

Obviously I can already search on any of those names (without using the field name) and find the doc as long as the other docs don't also contain the names or parts of names in other fields.

But if I want one field I can always search on I can create a scripted field like this which uses the first of the values it finds;

doc['name.keyword'].value ?: doc['firstName.keyword'].value ?: doc['fullName.keyword'].value ?: ''

Or this also works (might need one final return for the case where all three are null);

if (doc['name.keyword'].value != null)
  return doc['name.keyword'].value;
if (doc['firstName.keyword'].value  != null)
  return doc['firstName.keyword'].value;
if (doc['fullName.keyword'].value != null)
  return doc['fullName.keyword'].value;


How to check a doc field exist while creating scripted field using painless
Aggregate multiple records to single chart
Return value of null
Scripted field conditionals failing (5 of 15 shards failed)
(Chris Clifton) #3

Very cool! Thanks and I'll try this out.

(system) #5