Hello,
I have an ingest pipeline that has a simple enrich processor, it basically adds a field named host.type
based on the host.name
.
This enrich processor is part of a final_pipeline
that is set in the index, and I'm sure that this pipeline is executed, as the first processor creates a new field that is present in every document.
This is the final pipeline
{
"description": "crowdstrike final pipeline",
"processors": [
{
"pipeline": {
"name": "crowdstrike-set-company",
"ignore_failure": true
}
},
{
"enrich": {
"field": "host.name",
"policy_name": "crowdstrike-host-type",
"target_field": "enrich",
"ignore_missing": true,
"description": "adiciona 'enrich' de acordo com o tipo do host"
}
},
{
"rename": {
"field": "enrich.host.type",
"target_field": "host.type",
"ignore_missing": true,
"if": "ctx.enrich?.host?.type != null"
}
},
{
"remove": {
"field": "enrich.host.name",
"ignore_missing": true
}
},
{
"set": {
"field": "host.type",
"value": "UNKNOWN",
"override": false,
"if": "ctx.host?.type == null"
}
}
],
"on_failure": [
{
"set": {
"field": "error.message",
"value": "{{ _ingest.on_failure_message }}"
}
}
]
}
As you can see, I have a fail back to add the host.type
field if for some reason the enrich processor does not work or is not update with the inventory of hosts, we need this field to exist for our alerting system to work.
The policy for this enrich processor is the following:
{
"match": {
"indices": "inventory-crowdstrike",
"match_field": "host.name",
"enrich_fields": ["host.type"]
}
}
So, the documents in this .enrich-*
index will be something like this:
{
"_index" : ".enrich-crowdstrike-host-type-1643385722111",
"_type" : "_doc",
"_id" : "BMhsoX4BuJmOVklEI0aF",
"_score" : 9.393939,
"_source" : {
"host" : {
"name" : "REDACTED-HOSTNAME",
"type" : "WORKSTATION"
}
}
}
The issue is that sometimes it works, sometimes it does not work, for example, filtering for a host.name
that exists in the .enrich
index, I've got this in Kibana.
Any Idea on how to troubleshoot this? It seems that there is no failure in the pipeline, as the error.message
field is not created.
I'm on 7.16.3.