Testing on 7.6.2, the following code works fine in using a final_pipeline and a pipeline for the index. Using the same code and updated index template code, it actually appears to run twice on 7.15.2 (and 7.16.0).
The following example appends a value and the value shows up 2 times rather than just once after running through the pipeline
PUT _ingest/pipeline/test-final-pipeline
{
"processors": [
{
"append": {
"field": "pipeline",
"value": "completed"
}
}
]
}
PUT _template/test-template
{
"index_patterns": [
"test*"
],
"settings": {
"final_pipeline": "test-final-pipeline",
"number_of_shards": 2
}
}
POST /test-1/_doc
{
"time-field": "2021-02-10T12:02:01.789Z",
"CONTENT": "blah1"
}
PUT /_ingest/pipeline/routing-test-pipeline
{
"description": "Time series DAY pipeline",
"processors": [
{
"date_index_name": {
"field": "time-field",
"index_name_prefix": "test-",
"date_rounding": "d",
"index_name_format": "yyyy-MM-dd"
}
}
]
}
POST /test-1/_doc?pipeline=routing-test-pipeline
{
"time-field": "2021-02-10T12:02:01.789Z",
"CONTENT": "blah2"
}
The next example shows that trying to edit a field name, if you add ignore_missing: true
, the field name is already renamed:
PUT _ingest/pipeline/test-final-pipeline
{
"processors": [
{
"rename": {
"field": "CONTENT",
"target_field": "MALCONTENT"
}
}
]
}
PUT _template/test-template
{
"index_patterns": [
"test*"
],
"settings": {
"final_pipeline": "test-final-pipeline",
"number_of_shards": 2
}
}
POST /test-1/_doc
{
"time-field": "2021-02-10T12:02:01.789Z",
"CONTENT": "blah1"
}
PUT /_ingest/pipeline/routing-test-pipeline
{
"description": "Time series DAY pipeline",
"processors": [
{
"date_index_name": {
"field": "time-field",
"index_name_prefix": "test-",
"date_rounding": "d",
"index_name_format": "yyyy-MM-dd"
}
}
]
}
POST /test-1/_doc?pipeline=routing-test-pipeline
{
"time-field": "2021-02-10T12:02:01.789Z",
"CONTENT": "blah2"
}
Is there anything missing in these tests or could Issue #69727 or Issue #75047 be related to causing this to run twice when using the date math processor for the inde?