Kibana - access aggregation result

[fgbogdan (Fagadar-Ghisa Bogdan)]

i have an array of objects in elastic index and I need to display that into a Kibana visualization table.
Example - Parent-name, [ {Child-name, Child-age} ] (the row is the parent) and using Term aggregation for Parent-name and Child-name i get one line for each pair Parent - Child, but I cannot display the correct Child-age (Kibana display the same age for all Child of a parent)
(this is only a sample, the table structure is more complicated)
How can I access/display the correct Child-age ?
or
How can I access the value from an aggregation result from painless ? to iterate over the Child array and retrieve the correct Child-age.

LE: I cannot/want to change the index structure. Using nested fields is worst.

[LeeDr (Lee Drengenberg)]

What version of Kibana are you using? There might be some features available on the latest releases that help in this situation that aren't on older versions.

Are you using the default distribution or the oss distribution of Kibana?

If on default distribution are you using the built-in basic license, or something else?

If you could post a couple of sample docs in a format that could be run in the Kibana dev tools console that would make it much easier for someone to try to find a solution.

Regards,
Lee

[fgbogdan (Fagadar-Ghisa Bogdan)]

Hi, I use 7.5.0 default distribution on a linux machine (debian) as dashboard app for our product https://prolion.com/en/restoremanager, as basic license.
We store in elastic some info from files systems, and one of the fiels is an array of ACL's (access control list).
I need a report where to show for each file, the user and his specific rights. One possibility is to present all of them in a single field, using painless code and formatting the content like an array , into a table visualization.
Using only as aggregation term i get the correct visualization, meaning more lines for a file, one for each user from the ACL object. But i am unable to add the corresponding rights for each user.

the best that i got was something like {
"script": {
"inline": "def l = new ArrayList(); for (item in params._source.aces_object) { l.add(item.name + ' ' + item.sid); } return l;",
"lang": "painless"
}
}

but is still in one field and i want a different one for each user.

[fgbogdan (Fagadar-Ghisa Bogdan)]

The Es structure is:
{

• "state": "open",
• "settings": {
  • "index": {
    • "number_of_shards": "1",
    • "provided_name": "restoremanager_rw",
    • "creation_date": "1576658757810",
    • "analysis": {
      • "normalizer": {
        • "normalizer_case_insensitive": {
          • "filter": [
            • "lowercase"],
          • "type": "custom"}}},
    • "number_of_replicas": "1",
    • "uuid": "Dpk36ZyNT2KU8kTmKDAbOA",
    • "version": {
      • "created": "7050099"}}},
• "mappings": {
  • "snapdiff": {
    • "properties": {
      • "filetype": {
        • "type": "keyword"},
      • "volume_id": {
        • "type": "integer"},
      • "snapshot_id": {
        • "type": "integer"},
      • "snapshot_cr_date": {
        • "format": "yyyy-MM-dd HH:mm:ss",
        • "type": "date"},
      • "delta": {
        • "type": "long"},
      • "volume_name": {
        • "type": "keyword"},
      • "change_type": {
        • "type": "keyword"},
      • "file_end_date": {
        • "type": "long"},
      • "group_owner": {
        • "type": "text",
        • "fields": {
          • "keyword": {
            • "ignore_above": 256,
            • "type": "keyword"}}},
      • "mtime": {
        • "type": "long"},
      • "crtime": {
        • "type": "long"},
      • "inode": {
        • "type": "long"},
      • "shares": {
        • "properties": {
          • "share_name": {
            • "type": "keyword"}}},
      • "file_en_date": {
        • "format": "yyyy-MM-dd HH:mm:ss",
        • "type": "date"},
      • "ctime": {
        • "type": "long"},
      • "links": {
        • "type": "integer"},
      • "s3_uuid_key": {
        • "type": "text"},
      • "fattr": {
        • "type": "integer"},
      • "filesubtype": {
        • "type": "keyword"},
      • "group": {
        • "type": "long"},
      • "owner": {
        • "type": "long"},
      • "cluster_name": {
        • "type": "keyword"},
      • "atime": {
        • "type": "long"},
      • "user_owner": {
        • "type": "keyword"},
      • "level": {
        • "type": "integer"},
      • "aces_object": {
        • "properties": {
          • "allow": {
            • "type": "boolean"},
          • "right_Execute": {
            • "type": "boolean"},
          • "access": {
            • "type": "integer"},
          • "right_Read": {
            • "type": "boolean"},
          • "basic_right_Modify": {
            • "type": "boolean"},
          • "right_Write_EA": {
            • "type": "boolean"},
          • "flags": {
            • "type": "integer"},
          • "right_Delete": {
            • "type": "boolean"},
          • "basic_right_execute_list": {
            • "type": "boolean"},
          • "right_Generic_All": {
            • "type": "boolean"},
          • "basic_right_Read": {
            • "type": "boolean"},
          • "sid": {
            • "type": "keyword"},
          • "right_Synchronize": {
            • "type": "boolean"},
          • "right_Write_DAC": {
            • "type": "boolean"},
          • "right_Generic_Execute": {
            • "type": "boolean"},
          • "right_Delete_Child": {
            • "type": "boolean"},
          • "right_Read_EA": {
            • "type": "boolean"},
          • "right_Read_A": {
            • "type": "boolean"},
          • "basic_right_Write": {
            • "type": "boolean"},
          • "right_Write_Owner": {
            • "type": "boolean"},
          • "right_System_Security": {
            • "type": "boolean"},
          • "right_Generic_Read": {
            • "type": "boolean"},
          • "basic_right_Full_Control": {
            • "type": "boolean"},
          • "right_Read_Control": {
            • "type": "boolean"},
          • "right_Write": {
            • "type": "boolean"},
          • "name": {
            • "type": "keyword"},
          • "right_Write_A": {
            • "type": "boolean"},
          • "right_Append": {
            • "type": "boolean"},
          • "right_Generic_Write": {
            • "type": "boolean"}}},
      • "node_name": {
        • "type": "keyword"},
      • "fileext": {
        • "type": "text",
        • "fields": {
          • "keyword": {
            • "normalizer": "normalizer_case_insensitive",
            • "type": "keyword"}}},
      • "file_del_date": {
        • "type": "long"},
      • "host_id": {
        • "type": "integer"},
      • "s3_bucket_name": {
        • "type": "text",
        • "fields": {
          • "keyword": {
            • "ignore_above": 256,
            • "type": "keyword"}}},
      • "group_owner_sid": {
        • "type": "text",
        • "fields": {
          • "keyword": {
            • "ignore_above": 256,
            • "type": "keyword"}}},
      • "dos_bits": {
        • "type": "integer"},
      • "filename": {
        • "type": "text",
        • "fields": {
          • "keyword": {
            • "normalizer": "normalizer_case_insensitive",
            • "type": "keyword"}}},
      • "ftype": {
        • "type": "integer"},
      • "size": {
        • "type": "long"},
      • "svm_name": {
        • "type": "keyword"},
      • "user_owner_sid": {
        • "type": "keyword"},
      • "snapshot_create_date": {
        • "type": "long"},
      • "id_run": {
        • "type": "integer"},
      • "snapshot_uuid": {
        • "type": "text"},
      • "svm_id": {
        • "type": "integer"}}}},
• "aliases": ,
• "primary_terms": {
  • "0": 1},
• "in_sync_allocations": {
  • "0": [
    • "LMHOMsWqRymvhQ4sGIkvqg"]}

}

[fgbogdan (Fagadar-Ghisa Bogdan)]

and the best I can do is :

{
"script": {
"inline": "def l = new ArrayList(); for (item in params._source.aces_object) { l.add(item.name + ' ' + (item.basic_right_Modify?' Modify':'')+(item.basic_right_execute_list?' Exec/Lst':'')+(item.basic_right_Read?' Read':'')+(item.basic_right_Write?' Write':'')+(item.basic_right_Full_Control?' Full':'')); } return l;",
"lang": "painless"
}
}

[LeeDr (Lee Drengenberg)]

Sorry, your last posts came just before a long holiday break and this got lost in the wrapping paper.

There isn't a way in Kibana to represent one doc which has a field containing multiple values as multiple docs. I think the only way you might be able to do that is with the new Transforms that are part of the Machine Learning app of Kibana. See; https://www.elastic.co/webinars/introducing-data-frame-transforms-for-elastic-machine-learning

But even that might not do it.

If you did already find some solution please let us know.

Regards,
Lee

[fgbogdan (Fagadar-Ghisa Bogdan)]

No problem.
I used the solution above - processing the field and presenting the result as string, something like that:

It will be nice to structure the painless output - maybe as html or table, but for now it is impossible.

[Marius_Dragomir]

You can probably make the output nicer by using a scripted field.

[fgbogdan (Fagadar-Ghisa Bogdan)]

Yeah, interesting idea - will try later and share the result.
10x.