See Stackoverflow: http://stackoverflow.com/questions/42393543/elasticsearch-5-2-pipeline-can-not-access-nested-fields
I need to access nested fields in pipeline processors. It should be a frequent use case, however, I can not figure it out.
My mapping:
"properties" : {
"attachment" : {
"type" : "object",
"properties" : {
"content" : {"type" : "text", "store" : true, "index" : true, "analyzer" : "german" },
"title" : {"type" : "text", "store" : true},
"date" : {"type" : "date", "store" : true},
"author" : {"type" : "text", "store" : true},
"keywords" : {"type" : "keyword", "store" : true},
"content_type" : {"type" : "keyword", "store" : true},
"content_length" : {"type" : "integer", "store" : true},
"language" : {"type" : "keyword", "store" : true}
}
}
}
Pipeline configuration:
"description" : "Extract attachment information",
"processors" : [
{
"attachment" : {
"field" : "data",
"indexed_chars" : -1
}
},
{
"remove": {
"field": "data"
}
},
{
"split": {
"field" : "attachment.keywords",
"separator" : "\\s+"
}
}
]
If I put a non nested field, I works OK. With a nested field (regardless whether it is the subfield of attachment field or any other) it returns the following error message:
{"error":{"root_cause":[{"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [keywords] not present as part of path [attachment.keywords]","header":{"processor_type":"split"}}],"type":"exception","reason":"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [keywords] not present as part of path [attachment.keywords]","caused_by":{"type":"illegal_argument_exception","reason":"java.lang.IllegalArgumentException: field [keywords] not present as part of path [attachment.keywords]","caused_by":{"type":"illegal_argument_exception","reason":"field [keywords] not present as part of path [attachment.keywords]"}},"header":{"processor_type":"split"}},"status":500}
Please, help. I feel that it is quite natural to work further in a pipeline with the fields generated by the Ingest Attachment Processor.