# How to convert logstash elasticsearch plugin in ingest pipeline?

**URL:** <https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [June 27, 2021, 5:13pm UTC](https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156 "2021-06-27T17:13:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![crpj](https://avatars.discourse-cdn.com/v4/letter/c/edb3f5/32.png) [@crpj](https://discuss.elastic.co/u/crpj)\
**Post date:** [June 27, 2021, 5:13pm UTC](https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156/1 "2021-06-27T17:13:03Z")

</div>

Hello,

I would like to know if and how it is possible to use the logstash elasticsearch plugin in an ingest pipeline, currently the use of a workaround in logstash which enables enriching data based in a query from another index works fine as follows:

```auto
#logstash excerpt
filter {

    elasticsearch {
	  hosts => "http://localhost:9200/"
      query_template => "C:/Elastic/get_key_query.json"
      index => "ref-data-keys"
      fields => {
        "KEY" => "KEY"
      }
      remove_field => ["host", "@version", "@timestamp"]
    }
}
}
output {
	elasticsearch {
		hosts => "http://localhost:9200/"
		index => "destination-index-%{+YYYYMM}"
  }
}

```

```auto
#json query found in a separate file
{
  "size": 1,
  "_source": ["KEY"],
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "PRODUCT.keyword": "%{PRODUCT}"
          }
        },
        {
          "match": {
            "COMPONENT.keyword": "%{COMPONENT}"
          }
        }
      ]
    }
  }
}

```

It is possible to replicate this logic in ingest pipeline?

Thank you.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 27, 2021, 9:40pm UTC](https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156/2 "2021-06-27T21:40:16Z")

</div>

Hi @crpj

Yes there is there is even a special processor to enrich data in it ingest. See Here

> **[Enrich your data | Elasticsearch Guide \[7.13\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest-enriching-data.html)**

---

<div class="post-metadata">

**Author:** ![crpj](https://avatars.discourse-cdn.com/v4/letter/c/edb3f5/32.png) [@crpj](https://discuss.elastic.co/u/crpj)\
**Post date:** [June 28, 2021, 4:26pm UTC](https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156/3 "2021-06-28T16:26:53Z")

</div>

Thank you @stephenb

I have seen the Enrich feature once, if I understood it right, Enrich can only work with a single field for the lookup. Then I had the workaround I mentioned, the way I did with logstash and the elasticsearch plugin allows me to create a more complex query based in several fields (e.g. PRODUCT, COMPONENT and many more) to find a value (e.g. Key) that will be used to enrich the destination field.

My question is whether it is possible to replicate the same workaround I posted above directly in an ingest pipeline, I believe it is not possible because it references a json query file but wanted to confirm. The objective is to use Transforms and associate an ingest pipeline with that more complex enrich logic.

ps: that workaround was based in @dadoonet solution (thanks by the way)  
[https://david.pilato.fr/blog/2018/03/22/enriching-your-postal-addresses-with-elastic-stack-part-1/](https://david.pilato.fr/blog/2018/03/22/enriching-your-postal-addresses-with-elastic-stack-part-1/)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [June 28, 2021, 5:38pm UTC](https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156/4 "2021-06-28T17:38:41Z")

</div>

Well you could concatenate the the PRODUCT and COMPONENT into a single term field at ingest time, and use that as a lookup in the enrich, you would need to do that in the lookup data as well, I did that for a very similar case worked great... if you need to use a query then no, enrich processor is probably not the correct approach.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 26, 2021, 5:39pm UTC](https://discuss.elastic.co/t/how-to-convert-logstash-elasticsearch-plugin-in-ingest-pipeline/277156/5 "2021-07-26T17:39:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.