# Ingest pipline - multiple fields processed by one porcessor **URL:** \ **Category:** Elasticsearch\ **Created:** [February 28, 2019, 10:58am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320 "2019-02-28T10:58:38Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [February 28, 2019, 10:58am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/1 "2019-02-28T10:58:38Z")
Hi, Is there any solution how I could process multiple filds with a single processor using ingest node. For e.g. I would like to process source.ip dst.ip ip client.ip I would like t process all of them using geoip and output to source.ip.geo dst.ip.geo ip.geo client.ip.geo ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 28, 2019, 1:30pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/2 "2019-02-28T13:30:49Z")
I don't think you can. You need to run geoip processor 4 times I believe. ---
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [February 28, 2019, 2:21pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/3 "2019-02-28T14:21:57Z")
I have tried to run it twice but it is not accepting two instances of geoip ``` PUT _ingest/pipeline/geoip-info { "description": "Add geoip info", "processors": [ { "geoip": { "field": "ip", "target_field": "client.geo", "ignore_failure": true }, "geoip": { "field": "source.ip", "target_field": "sourceip.geo", "ignore_failure": true } } ] } ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 28, 2019, 2:50pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/4 "2019-02-28T14:50:00Z")
Ha right! I think you can use this then: [https://www.elastic.co/guide/en/elasticsearch/reference/current/foreach-processor.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/foreach-processor.html) ---
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [February 28, 2019, 2:55pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/5 "2019-02-28T14:55:48Z")
Thank you - that's excellent ---
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [February 28, 2019, 3:36pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/6 "2019-02-28T15:36:52Z")
David, one more question. foreach needs to receive an array field. Supposedly I want to process two fields source.ip and dest.ip - how do I provide it for foreach processor? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 2, 2019, 11:38am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/7 "2019-03-02T11:38:36Z")
Sorry. I was wrong. There is a more obvious way to make it work: ```auto POST _ingest/pipeline/_simulate { "pipeline": { "description": "date pipeline ", "processors": [ { "geoip": { "field": "dest", "target_field": "dest_geoip" } }, { "geoip": { "field": "source", "target_field": "source_geoip" } } ] }, "docs": [ { "_index": "index", "_type": "_doc", "_id": "id", "_source": { "dest": "80.34.121.50", "source": "80.34.121.50" } } ] } ``` ---
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [March 2, 2019, 10:36pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/8 "2019-03-02T22:36:54Z")
David, Thank you for this again - it seams to be a solution. However, Packetbeat returns fields in dot notation like dest.ip source.ip - once I use those fields - pipeline is failing - is there any way to convert dot notation in to something else? It would have to be done on Packetbeat level - unless I can convince ingest node to process dot notation fields. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 2, 2019, 10:47pm UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/9 "2019-03-02T22:47:42Z")
Have a look at the rename processor. ---
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [March 3, 2019, 12:02am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/10 "2019-03-03T00:02:43Z")
David, Fields with dot notation cannot be processed by any processor according to documentation. dot\_expander processor is a solution. [https://www.elastic.co/guide/en/elasticsearch/reference/6.6/dot-expand-processor.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/dot-expand-processor.html) " Expands a field with dots into an object field. This processor allows fields with dots in the name to be accessible by other processors in the pipeline. Otherwise these [fields](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/accessing-data-in-pipelines.html) can’t be accessed by any processor." I have tried rename but it had problem with dot notation also. Below is a final pipeline which allow multiple use of the same processor and also allow processing of fields with dot notations like source.ip dest.ip as they come from Packetbeat. ``` POST _ingest/pipeline/_simulate { "pipeline": { "description": "date pipeline ", "processors": [ { "dot_expander": { "field": "source.ip" } }, { "dot_expander": { "field": "dest.ip" } }, { "geoip":{ "field":"source.ip", "target_field":"source_geoip" } }, { "geoip":{ "field":"dest.ip", "target_field":"dest_geoip" } } ] }, "docs": [ { "_index": "index", "_type": "_doc", "_id": "id", "_source": { "dest.ip": "80.34.121.50", "source.ip": "191.12.41.50" } } ] } ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 3, 2019, 3:15am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/11 "2019-03-03T03:15:49Z")
Great! Thanks for sharing the final pipeline. (I was not aware of the `dot_expander` processor πŸ˜‰) ---
**Author:** ![derekmizak](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Post date:** [March 3, 2019, 3:26am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/12 "2019-03-03T03:26:28Z")
Hey - it was discovery for me as well πŸ˜‰ ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 31, 2019, 3:26am UTC](https://discuss.elastic.co/t/ingest-pipline-multiple-fields-processed-by-one-porcessor/170320/13 "2019-03-31T03:26:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.