Using painless with aggregation results


(Sri) #1

Hi,

I am a little new to using painless scripting. I was able to use it in a kibana visualization to convert the units of a field. Also in a query_and_update scenario.

Now I have a situation/aggregation that does not fit kibana, so trying to use the 3rd party transform plugin. I thought it should be possible to use scripted_fields to count the number of occurrences of a specific term in the aggregated results. But, I havent figured how the right syntax it seems.

Once past this problem - If statusText can be "a" "b" or "c" etc., I would like to calculate how many "a", "b" and "c" were found.

{
  "aggs": {
      "terms": {
        "field": "myfield.raw",
        "size": 10000
      },
      "aggs": {
        "latest_record": {
          "top_hits": {
            "sort": [
              {
                "@timestamp": {
                  "order": "desc"
                }
              }
            ],
            "_source": {
              "includes": [
                "statusText"
              ]
            },
            "size": 1
          }
        }
      }
    },
    size = 0,
   "scripted_fields": {
   "script": {
		"lang": "painless",
		"inline": "int cnt=0; return cnt;"
      }
   }
}

"[parsing_exception] Unknown key for a START_OBJECT in [scripted_fields]., with { line=1 & col=227 }"

Thanks


(Simon Willnauer) #2

I think your scripted fields should look like this:

"scripted_fields": {
   "my_field_name" : {
      "script": {
	  	"lang": "painless",
		"inline": "int cnt=0; return cnt;"
        }
   }
}


(Sri) #3

Thanks @s1monw Simon,

I tried the additional "nesting" now and got the same error. I believe I had tried few variants like this earlier too.

{"error":{"root_cause":[{"type":"parsing_exception","reason":"Unknown key for a START_OBJECT in [scripted_fields].","line":1,"col":227}],"type":"parsing_exception","reason":"Unknown key for a START_OBJECT in [scripted_fields].","line":1,"col":227},"status":400}

Could it be the type of aggs query in this specific example are not compatible? I have confirmed the aggs query part alone is returning results.

I'm ok to try if you have another working aggs example I could try in my env to confirm it works.


(Simon Willnauer) #4

oh well :smiley: it's script_fields not scripted_fields I am sorry I didn't see it earlier.


(Sri) #5

Oh no! cant believe I was blind :cry:

I'm still not exactly clear how script_fields works with aggregations - where does the script execute & how would response change? I was expecting a "my_field_name" added to the aggs response, but I dont see any change to the response with or without the script_fields section. I've gone through the docs and tried to understand the examples I saw in the forums, but would be good if there are other references you know.