Hi. I am attempting to use nested values in a script score, but I am having issues making it work, because I am unable to iterate over the field by accessing it through doc. Also, when I try to query it in Kibana like _type:images AND _exists_:colors
, it will not match any documents, even though the field is clearly present in all my docs when I view them individually. I am however able to access it using params._source, but I have read that it can be slow slow and is not really recommended.
I know that this issue is all due to the way we have created this nested field, so if I cannot come up with something better than this, I will have to reindex our 2m+ documents and see if I can find another way around the problem, but I would like to avoid that, and also just get a better understanding of how Elastic works behind the scenes, and why it acts the way it does here.
The example I will provide here is not my real life issue, but describes the issue just as well.
Imagine we have a document, that describes an image. This document has a field that contains values for how much red, blue, and green exists in an image.
Requests to create index and documents with nested field that contains arrays of colors with a 100 point split between them:
PUT images
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"_doc": {
"properties": {
"id" : { "type" : "integer" },
"title" : { "type" : "text" },
"description" : { "type" : "text" },
"colors": {
"type": "nested",
"properties": {
"red": {
"type": "double"
},
"green": {
"type": "double"
},
"blue": {
"type": "double"
}
}
}
}
}
}
}
PUT images/_doc/1
{
"id" : 1,
"title" : "Red Image",
"description" : "Description of Red Image",
"colors": [
{
"red": 100
},
{
"green": 0
},
{
"blue": 0
}
]
}
PUT images/_doc/2
{
"id" : 2,
"title" : "Green Image",
"description" : "Description of Green Image",
"colors": [
{
"red": 0
},
{
"green": 100
},
{
"blue": 0
}
]
}
PUT images/_doc/3
{
"id" : 3,
"title" : "Blue Image",
"description" : "Description of Blue Image",
"colors": [
{
"red": 0
},
{
"green": 0
},
{
"blue": 100
}
]
}
Now, if I run this query, using doc:
GET images/_search
{
"query": {
"function_score": {
"functions": [
{
"script_score": {
"script": {
"source": """
boolean debug = true;
for(color in doc["colors"]) {
if (debug === true) {
throw new Exception(color["red"].toString());
}
}
"""
}
}
}
]
}
}
}
I will get exception No field found for [colors] in mapping with types []
, but if I use params._source instead, like so:
GET images/_search
{
"query": {
"function_score": {
"functions": [
{
"script_score": {
"script": {
"source": """
boolean debug = true;
for(color in params._source["colors"]) {
if (debug === true) {
throw new Exception(color["red"].toString());
}
}
"""
}
}
}
]
}
}
}
I am able to output "caused_by": {"type": "exception", "reason": "100"}
, so I know that it worked since the first document is "Red Image" and therefore has a red value of 100.
I am not even sure that this can classify as a question, but more a cry for help. If someone can explain why this is behaving the way it is, and give an idea of the best way to get around the issue, I would really appreciate it.
(Also, some tips for debugging in Painless would also be lovely!!!)