Aggregate over an array

Elastic Stack Kibana
1

I have an array field,

strArray: ['browser:IE', 'device:PC', 'country:USA', 'state:CA']

I'd like to draw a bar chart in kibana showing the most common entries in position 0 of that field.

I'm entering into the Advanced JSON input :

{
   "terms": {
       "script": "doc['strArray']"
    }
}

but my queries fail. What am I missing?

2

As far as I know, Elasticsearch doesn't provide a way to do an aggregations on a position in the array, just on all the values it finds. Internally, I don't think Elasticsearch keeps track of the order of the items at all, so there's not even a way to make this work.

Your best bet is, if you know that the position of the values in that array have an important meaning, is to simply index that value in a new field along side that strArray field.

3

Interesting. Is there no notion of position for arrays in ES or is it just not a built in aggregation?

Would it be possible to write a (groovy?) script that somehow gets around this?

4

You could probably do a scripted agg for this.

5

I'm not well versed in groovy scripting in ES, but you might be able to do this via a scripted field. Here's the docs if you want to give it a go. Be aware that you have to manually enable groovy scripting in your configuration.

I was told that Elasticsearch doesn't ensure the order of arrays though, so it's possible this may not work either. I may have been told wrong though, asking about it over on the elasticsearch boards would probably get you better answers.

6

scripted aggs are possible in kibana, right?

7

We support scripted fields using the Lucene Query Syntax out of the box. If you want to use groovy scripts, you'll need to enable them in elasticsearch first (be aware that there are security implications).

You can use scripted aggs should work too, using the JSON input and crafting your query by hand that way.

8

Is there an example of scripted aggs somewhere?

1 Like
9

I actually don't know about any examples. There are probably some examples on the Internet somewhere if you go look for them though.

If you're asking about scripted metric aggs, I don't think they are possible in Kibana. The JSON input simply lets you add additional parameters to the existing aggregation, and we don't offer scripted metrics as an option.

You might be able to do what you're trying to do with scripted fields though. Scripted fields are different, they are added as fields to your index pattern. Out of the box, the are limited to the Lucene syntax, which only supports numbers. But, if you enable groovy scripting, you can give you field a custom script type. See the script fields, scripts in terms aggs and scripting docs to get started.