Using two enrich policies in one ingestion pipeline

Elastic Stack Elasticsearch
1

Hi,

I'm trying to enrich one index by applying two enrich policies with two different source indices.
Here is my approach:
"enrich":
[
{
"policy_name": "vehicle-database-policy",
"field" : "vehicle.id",
"target_field": "vehicle-db",
"max_matches": "1"
},
{
"policy_name": "axle-database-policy",
"field" : "axle.id",
"target_field": "vehicle-db",
"max_matches": "1"
}

Using just one enrich policy is working but as soon as I try to use the second policy as well i get this error:

{
"error" : {
"root_cause" : [
{
"type" : "parse_exception",
"reason" : "property isn't a map, but of type [java.util.ArrayList]",
"processor_type" : "enrich"
}
],
"type" : "parse_exception",
"reason" : "property isn't a map, but of type [java.util.ArrayList]",
"processor_type" : "enrich"
},
"status" : 400
}

Any help is appreciated!
Thx!

2

Have you tried using two seperate enrich processors?

3

Yep! Duplicate keyword enrich... Not working

4

please share your full pipeline as a formatted snippet.

5

PUT /_ingest/pipeline/solution
{
"description" : "split and index csv files from solution",
"processors" :
[
{
"csv":
{
"field": "message",
"target_fields":[
"@timestamp",
"lots of fields.....",
"even more fields"
]
},
"enrich":
[
{
"policy_name": "vehicle-database-policy",
"field" : "vehicle.rfid.id",
"target_field": "vehicle-db",
"max_matches": "1"
},
{
"policy_name": "axle-database-policy",
"field" : "axle.axle-position",
"target_field": "vehicle-db",
"max_matches": "1"
}
],
"set" :
{

      "field" : "site.geoloc_point",
      "value" : "{{site.geoloc.lat}}, {{site.geoloc.lon}}"
    }
}

],
"on_failure" : [
{
"set" : {
"field" : "error",
"value" : " - Error processing message - "
}
}
]
}

6

This is still using a single enrich processor. Try

processors: [
  {
    "enrich" : {}
  },
  {
    "enrich" : {}
  }
]
7

It's not working but at least there is no error. So I think it is on my side now! That's progress!

Thx!

8

feel free to share the output of the Simulate Pipeline API with some sample documents. Do not forget the verbose parameter for maximum debugging.

9

Some minutes of head scratching and in depth comparison i found the bug. A "_" instead of an "-" within a field name!

Now it is working as intended!
Thx for your time!

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.