# Ingest Pipeline for Transformations

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736>\
**Category:** Elasticsearch\
**Created:** [July 10, 2020, 5:46pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736 "2020-07-10T17:46:24Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![AV25242](https://avatars.discourse-cdn.com/v4/letter/a/8e8cbc/32.png) [@AV25242](https://discuss.elastic.co/u/AV25242)\
**Post date:** [July 10, 2020, 5:46pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/1 "2020-07-10T17:46:24Z")

</div>

I am very new to elastic search, I have downloaded Elastic search on MacOs and have also installed Kibana for visualization. I am working with a connector which allows me to provide a pipeline name. However I dont know how to create an Ingest Pipeline in Elastic search.

For instance if my incoming data has a field called Name and I want to make it all UpperCase  
Where do I create a pipeline and its processors. I read this [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html) but couldnot follow as to where should I be creating a pipeline, i have access to ES on port 9200, is there a file that I need to create? Can someone point me to an example on creating a pipeline, processor and consuming it.  
Also are there any pipelines with processor already available ?

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [July 10, 2020, 5:56pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/2 "2020-07-10T17:56:09Z")

</div>

You can use the [put pipeline api](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-pipeline-api.html#put-pipeline-api)

Many of the filebeat modules load ingest pipelines with their "setup" phase. On any system where you have installed filebeat, do a "locate pipeline.yml" (that command is RH based, may be different on your OS) and you will see sample pipelines in many _/filebeat/module/_ directories.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 10, 2020, 6:12pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/3 "2020-07-10T18:12:22Z")

</div>

Welcome!

Open Kibana and go to the Dev Console.  
There you can copy paste the examples available in the documentation.

Click on play/run.

---

<div class="post-metadata">

**Author:** ![AV25242](https://avatars.discourse-cdn.com/v4/letter/a/8e8cbc/32.png) [@AV25242](https://discuss.elastic.co/u/AV25242)\
**Post date:** [July 11, 2020, 4:28pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/5 "2020-07-11T16:28:51Z")

</div>

I installed filebeat and I do see pipeline.yml, thats definitely helpful. This gives me ideas on how to write a pipeline that has processors. Now say I create my own pipeline.yml file, where should I deploy this file within elastic search such that ES respects it and uses it as a preprocessor before creating an Index ? Is there a specific folder structure inside ES where I can deploy this file that I have created ?

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [July 11, 2020, 5:29pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/6 "2020-07-11T17:29:48Z")

</div>

That's the "put pipeline api" above. Either command line (curl) or Kibana tools (easier).

Pipelines (and most everything else) are stored in the Elasticsearch database, so you only have to update once and it's available across the cluster.

---

<div class="post-metadata">

**Author:** ![AV25242](https://avatars.discourse-cdn.com/v4/letter/a/8e8cbc/32.png) [@AV25242](https://discuss.elastic.co/u/AV25242)\
**Post date:** [July 11, 2020, 5:52pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/7 "2020-07-11T17:52:57Z")

</div>

ah ok thats it 🙂 thanks much let me try one and will let you know

---

<div class="post-metadata">

**Author:** ![AV25242](https://avatars.discourse-cdn.com/v4/letter/a/8e8cbc/32.png) [@AV25242](https://discuss.elastic.co/u/AV25242)\
**Post date:** [July 13, 2020, 7:11pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/8 "2020-07-13T19:11:27Z")

</div>

@rugenl that worked like a charm thanks much.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 10, 2020, 7:11pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-transformations/240736/9 "2020-08-10T19:11:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
