# Enrich policy not working in logstash pipeline
**URL:** https://discuss.elastic.co/t/enrich-policy-not-working-in-logstash-pipeline/368026
**Category:** Elasticsearch
**Tags:** ingest-pipeline
**Created:** [September 30, 2024, 2:35pm UTC](https://discuss.elastic.co/t/enrich-policy-not-working-in-logstash-pipeline/368026 "2024-09-30T14:35:10Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Seemant\_Bind](https://discuss.elastic.co/u/Seemant_Bind)
#### Post date: [September 30, 2024, 2:35pm UTC](https://discuss.elastic.co/t/enrich-policy-not-working-in-logstash-pipeline/368026/1 "2024-09-30T14:35:10Z")
Hi,
I am trying to use enrich policy for indexing the data. I have 2 indices, one is emp\_master and another emp\_salary. My emp\_salary sheet will be updated frequently and logstash pipline will capture the information and push into elasticsearch. I want emp\_salary to be enriched with the emp\_master information. Below is the detail for the emp\_master index-
```auto
PUT /emp_master
{
"mappings": {
"properties": {
"emp_id": {
"type": "integer"
},
"emp_name": {
"type": "text"
}
}
}
}
POST /emp_master/_bulk
{ "index": { "_index": "emp_master" } }
{ "emp_id": 1, "emp_name": "krishna" }
{ "index": { "_index": "emp_master" } }
{ "emp_id": 2, "emp_name": "seemant" }
{ "index": { "_index": "emp_master" } }
{ "emp_id": 3, "emp_name": "arindam" }
```
In my emp\_salary index I have the below json data which I'm pushing through logstash-
```auto
{"emp_id":1, "salary":"11"}
{"emp_id":2, "salary":"12"}
{"emp_id":3, "salary":"13"}
```
On the basis of above, I want to get the emp\_name in my emp\_salary index for which I have created the below enrich policy and ingest pipeline-
```auto
PUT /_enrich/policy/emp_enrich
{
"match": {
"indices": "emp_master",
"match_field": "emp_id",
"enrich_fields": ["emp_name"]
}
}
#Execute enrich policy
PUT /_enrich/policy/emp_enrich/_execute
#Create ingest pipeline
PUT /_ingest/pipeline/emp_ingest_pipeline
{
"processors": [
{
"enrich": {
"policy_name": "emp_enrich",
"field": "emp_id",
"target_field": "employee",
"max_matches": 1
}
}
]
}
```
Below is my emp\_salary logstash conf -
```auto
input {
file {
path => "C:/sdas/asda/fsafa/emp_salary.json"
start_position => "beginning"
codec => "json"
type => "emp_salary"
add_field => { "source" => "emp_salary" }
sincedb_path => "NUL" # Forces Logstash to process all files from the beginning
}
}
filter {
if [type] == "emp_salary" {
json {
source => "message"
target => "parsed_json_emp_salary"
}
if ([parsed_json_emp_salary][emp_id]) {
mutate {
add_field => { "emp_id" => "%{[parsed_json_emp_salary][emp_id]}" }
}
}
if ([parsed_json_emp_salary][salary]) {
mutate {
add_field => {"salary" => "%{[parsed_json_emp_salary][salary]}" }
}
}
mutate {
copy => { "emp_id" => "[@metadata][_id]" }
}
}
}
output {
if [type] == "emp_salary" {
elasticsearch {
hosts => ["http://localhost:9200"]
user => ""
password => ""
index => "emp_salary"
document_id => "%{[@metadata][_id]}"
doc_as_upsert => true
action => "update"
pipeline => "emp_ingest_pipeline"
}
stdout { codec => rubydebug }
}
}
```
After doing the above, I am not getting the **emp\_name** field in the **emp\_salary** index.
However when I use `POST emp_salary/_update_by_query?pipeline=emp_ingest_pipeline` , I am getting the emp\_name in the index. Kindly see the screenshot below-
Am I missing anything to set up the pipeline or policy?
Can you please suggest how enrich policy can be applied in logstash pipeline?
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [September 30, 2024, 3:23pm UTC](https://discuss.elastic.co/t/enrich-policy-not-working-in-logstash-pipeline/368026/2 "2024-09-30T15:23:35Z")
Can you confirm if `emp_salary` is a normal index or if it was created as data stream?
I would expect this to work if this is a normal index.
---
### Author:  [@Seemant\_Bind](https://discuss.elastic.co/u/Seemant_Bind)
#### Post date: [October 1, 2024, 11:25am UTC](https://discuss.elastic.co/t/enrich-policy-not-working-in-logstash-pipeline/368026/3 "2024-10-01T11:25:51Z")
emp\_salary is a normal index
---
### Author:  [@Seemant\_Bind](https://discuss.elastic.co/u/Seemant_Bind)
#### Post date: [October 28, 2024, 1:43pm UTC](https://discuss.elastic.co/t/enrich-policy-not-working-in-logstash-pipeline/368026/4 "2024-10-28T13:43:20Z")
Hi @leandrojmp,
Apology for tagging you directly but there has been an production issue.
I have used doc\_as\_upsert =\> true action =\> "update" in the pipeline but when I removed this, it is working fine.
Could this be an issue for enrichment? For my case, I must have doc\_as\_upsert =\> true action =\> "update" in conf, can you provide some insight here how to enrich the data keeping those parameter enabled?