Kibana data table visualization and list of arrays question

Hello!

Our antispam gateway is pushing metadata to our elasticsearch cluster.

A pushed doc json looks like this:

{
   "rspamd_meta":{
      "rspamd_server":"out-mta01",
      "message_id":"cf1a3fdd58f966a299e1b820d67ee9f5@email.bi",
      "header_to":[
         "email@email.com"
      ],
      "asn":{
         "country":"CH",
         "asn":"49xx7",
         "ipnet":"94.xx.xx.0/20"
      },
      "webmail":false,
      "ip":"94.103.96.175",
      "header_from":[
         "\"Bienvenue sur le site\" <some.sender@email.bi>"
      ],
      "from":"someuser@web15.ourservers.com",
      "rcpt":[
         "email@email.com"
      ],
      "score":100,
      "user":"someuser",
      "direction":"Outbound",
      "hostname":"web15.ourservers.com",
      "symbols":[
         {
            "group":"ungrouped",
            "groups":[

            ],
            "options":[
               "email@email.com"
            ],
            "name":"BLACKLISTED_RCPT",
            "score":100
         },
         {
            "group":"ungrouped",
            "options":[
               "asn:49457, ipnet:94.x.x.0/20, country:CH"
            ],
            "name":"ASN",
            "score":0
         }
      ],
      "action":"reject",
      "is_local":false,
      "header_date":[
         "Tue, 31 Mar 2020 18:12:25 +0200"
      ],
      "header_subject":[
         "Bienvenue sur le site: Hey, wie ist deine Woche?"
      ],
      "qid":"6E452386-3BBF-4267-A1B6-08FB0FCC373C.1"
   },
   "@timestamp":"1585671145930.2"
}

So rspamd_meta.symbolsis pushed as an array of list (or the reverse i'm not really sure here)

The index pattern for the documents looks like:

In discover the result is:

What I am trying to do is, in our data table visualization, add a column that reports the symbols.name with the highest symbols.score.

I tried to do this adding a split row bucket on rspamd_meta.symbols.name limited to 1 result and sorted on custom metric based on rspamd_meta.symbols.score descending, but it's not giving the expected results.

Am I doing it wrong? Any idea how I could achieve this ?

Thanks a lot for any help!

Update: After a bit of reading, I tried to set the field type to "flattened" and reindexing all indices, but it's even worse. Kibana now shows the field as type "unknown".
I guess it's not the right path to follow to achieve what I'm trying to do :slight_smile:

Kibana does not supported nested for flattened fields. It may be best to break the array into individual documents

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.