Painless script - check for null

[bluemalkin (Tom M)]

Hi,

I have a painless script custom field using regex:

if (doc['message.keyword'].value != null) {
def m = /((GET|POST|PUT)\s{1}(/v1/[\w/-]+))(?|\s){1}/.matcher(doc['message.keyword'].value);
if (m.find()) { return m.group(1) } else { return "No match" }
}
else { return "NULL"}

It works mostly but I also get lots of messages which return NULL even though there IS a message.
For example this returned NULL

message: x.x.x.x - - [02/Mar/2018:13:42:10 +1100] "GET /v1/search/products?
api_key=*********** HTTP/1.1" 200 7952 "-" "Java/1.8.0_31 RestSDK/1.4.5"

Any idea why it's not matching doc['message.keyword'].value != null ?

Thanks,

[rjernst (Ryan Ernst)]

Are you escaping the / inside the regex? I'm surprised that does not throw a syntax error...

[bluemalkin (Tom M)]

Sorry I am escaping the / it's the rendering here that removes them

[rjernst (Ryan Ernst)]

What are the mappings for your index? You should only get null there if no values exist for the document for that field, but that could happen if you have docvalues disabled. What version of elasticsearch is this?

[bluemalkin (Tom M)]

I'm using the docker ES version 6.2.1

Mapping: https://pastebin.com/hw9fuK3T

It returns null for items that do have a docvalue "messages", very strange.

[rjernst (Ryan Ernst)]

Are you sure every document has a value for the field? Also, in your example before, your field was message.keyword, but in your last response it was messages. Can you run an exists query for the field name you are using and compare the total hits with those from a match all query?