Kibana visualization - Count occurence of term in multiple fields

[Ilya_Geller]

Hello,
I have index that containts documents with fields: source_address and destination_address for each event.

I would like to create a visualization in Kibana to show the top 10 most popular addresses in the database, both source and destination together.
To clarify, I do not mean create a chart for top 10 source addresses and a separate one for destination, but instead count how many times an address has occured at all (either in source or destination) and display the top values of that.

How can this be achieved?

Thanks.

[flash1293 (Joe Reuter)]

You can't do this purely with aggregations, but there are two options:

• Create a scripted field which returns an array of source and destination address: https://www.elastic.co/guide/en/kibana/current/scripted-fields.html Then use "unique count" on this scripted field
• If the first option is too slow, do the same thing at ingest time by using copy_to in your mapping: https://www.elastic.co/guide/en/elasticsearch/reference/current/copy-to.html

For the second option you have to reindex your existing data, but it will scale better if you have large amounts of data

[Ilya_Geller]

Thank you for the reply.

Won't "unique count" over this array field count the unique occurence of the pair [src,dst] together? I would like to achieve the count of docs where src=X or dest=X , not src=X and dest=Y , if this makes sense.

[flash1293 (Joe Reuter)]

No, Elasticsearch will treat each value of the array separately. You can simply use the "terms" aggregation to get the list of the top addresses:

POST myindex/_doc
{
  "myField": ["a", "b"]
}

POST myindex/_doc
{
  "myField": ["b", "c"]
}

POST myindex/_doc
{
  "myField": ["c", "d"]
}

GET myindex/_search?size=0
{
  "aggs": {
    "terms": {
      "terms": {
        "field": "myField.keyword",
        "size": 2
      }
    }
  }
}

// Result
      "buckets" : [
        {
          "key" : "b",
          "doc_count" : 2
        },
        {
          "key" : "c",
          "doc_count" : 2
        }
      ]

[Ilya_Geller]

That did the trick. Thank you!