New to Painless, can someone please help?

[wardallen]

I'm brand new to painless, so please forgive me if I'm asking a dumb question here.

I have an index that has events with many fields in them, and need to calculate a field based on values found in others, and include this field in the result.
Thing is I've never done any Painless before, and I'm hoping someone can help me out here.

The logic I need to implement is :

If field_A = "-" or ""
  if field_B = "YES" or "MAYBE"
	My_calculated_field = "OK"
  else
	My_calculated_field = "Not OK"
else
    My_calculated_field = "Not OK"

If field_A has rubbish data in it (Empty or just a - ) and field_B is one of two particular values, create a field that has "OK" in it.
The field should always be created for every document.

Here's what I have so far, and I'm not getting anywhere with it.

GET /logstash*/_search
{ 
  "query": {
    "regexp": {
      "field_C": "avalueIamlookingfor"
    }
  },
  "script_fields": {
    "My_calculated_field": {
      "script": {
        "lang": "painless",
        "source": """
        if (doc['field_A'].value == "-") OR  (doc['field_A'].value == "") {
          if (doc['field_B'].value == "YES") OR (doc['field_B'].value == "MAYBE") {
            My_calculated_field = "OK";
          } 
          else {
            My_calculated_field = "Not OK";
          } 
        } else {
            My_calculated_field = "Not OK";
          }
          """
        }
      }
    }
  }

Which results in :
Unhandled Exception illegal_argument_exception

unexpected token ['{'] was expecting one of [{, ';'}]. and points at the last ) in the doc[field_A] line.

What am I doing wrong?
And I can do aggregations on My_calculated_field once it's working, can't I?

[val (Val Crettaz)]

Great start!! You just have unbalanced parenthesis and OR should be || instead:

if (doc['field_A'].value == "-" || doc['field_A'].value == "") {
    if (doc['field_B'].value == "YES" || doc['field_B'].value == "MAYBE") {
        My_calculated_field = "OK";
    } else {
        My_calculated_field = "Not OK";
    }
} else {
    My_calculated_field = "Not OK";
}

[wardallen]

Thanks, Val.

Do I need to define a variable to return a value from the script?

I tried this

string My_calculated_field
if (doc['field_A'].value == "-" || doc['field_A'].value == "") {
    if (doc['field_B'].value == "YES" || doc['field_B'].value == "MAYBE") {
        My_calculated_field = "OK";
    } else {
        My_calculated_field = "Not OK";
    }
} else {
    My_calculated_field = "Not OK";
}
return My_calculated_field

But now I'm getting error messages saying variable [doc] is not defined.

Have you got any good links about this kind of stuff? I've never been a Java person, it seems there's some fundamentals I should know.

[spinscale (Alexander Reelsen)]

hey,

a) can you share the full request on your side with the most recent script?
b) I think you can shortcut the script quite a bit

def isA = doc['field_A'].value == "-" || doc['field_A'].value == "");
def isB = doc['field_B'].value == "YES" || doc['field_B'].value == "MAYBE";
return  isA && isB ? "OK" : "Not Ok"

The main question why doc does not exist is about the context in which this script is called, thus my question about the full request including the script.

--Alex

[psramkumar (Ramkumar Pillai)]

http://groovyconsole.appspot.com/
write your script here and test then add it

use groovy syntax almost same tp plainless

[wardallen]

Apologies for the slow reply, life got in the way

GET /logstash*/_search
{ 
  "query": {
    "regexp": {
      "field_C": "avalueIamlookingfor"
    }
  },
  "script_fields": {
    "My_calculated_field": {
      "script": {
        "lang": "painless",
        "source": """
        string My_calculated_field
        if (doc['field_A'].value == '-' || doc['field_A'].value == '') {
          if (doc['field_B'].value == "YES" || doc['field_B'].value == "MAYBE") {
            My_calculated_field= "OK";
          } 
          else {
            My_calculated_field= "OK";
          } 
        } else {
            My_calculated_field= "Not OK";
          }
          return My_calculated_field
        """
        }
      }
    }
  }

Something else occurred to me, I could also probably do this in logstash with grok, creating the field at ingestion time.
I'm thinking query time is better in a CPU usage sense, as the processing will be applied on a far smaller number of documents. Would you agree with that?

[Christian_Dahlqvist]

Doing it at index time is often the better option as you do it once for each document instead of once for every query. It also speeds up queries and scales better.

[wardallen]

Thanks for that. Unfortunately, I'm just as new at grok, and my attempts at implementing this don't seem to be working out. Are you able to help me figure out what I'm doing wrong?

  if ([field_A] in ["-", ""]) and ([field_B] in ["YES", "MAYBE"]) {
	mutate { add_field { "My_calculated_field" => "OK" } }
 } else {
           mutate { add_field { "My_calculated_field" => "Not OK" } } 
}

I keep getting errors when trying to start logstash, an excerpt :

:message=>"Expected one of [ \\t\\r\\n], \"x\", \"=>\" at line 33

Leading me to think that I've missed a closing } somewhere, but I can't see it. Can you?