# Multiple Pipelines with condition **URL:** \ **Category:** Elasticsearch\ **Created:** [November 3, 2023, 7:55pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405 "2023-11-03T19:55:27Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Manasa4](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@Manasa4](https://discuss.elastic.co/u/Manasa4)\ **Post date:** [November 3, 2023, 7:55pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405/1 "2023-11-03T19:55:27Z")
Hi Team, I have been trying to add a condition on my multi processor pipeline. { "4modelprocessor\_peopleagg": { "processors": [ { "pipeline": { "name": "ner\_pipeline\_peopleagg" } }, { "pipeline": { "name": "elser\_pipeline\_peopleagg" } } ] } } I should run the above elser or ner pipelines only when **resumecontent** or **aboutme** fields are available. Is there a way we can specify a condition to run the pipeline if one of the fields are available. Can someone please help me with the reference on this. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [November 3, 2023, 8:02pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405/2 "2023-11-03T20:02:49Z")
> [@Manasa4](#): > > Is there a way we can specify a condition to run the pipeline if one of the fields are available. > > Can someone please help me with the reference on this. Yes, you can conditionally run a processor, please check the [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html#conditionally-run-processor). ---
**Author:** ![Manasa4](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@Manasa4](https://discuss.elastic.co/u/Manasa4)\ **Post date:** [November 3, 2023, 9:20pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405/3 "2023-11-03T21:20:31Z")
Thanks for that. But I'm working on the inference processor. As per the documentation I can see it for set and drop processors. How can add my condition for the below inference processor : { "ner\_pipeline": { "processors": [ { "inference": { "model\_id": "dslim\_\_bert-base-ner", "target\_field": "ml.ner\_resume\_content", "field\_map": { "resumecontent": "text\_field", "ignore\_missing": "true" } } }, { "inference": { "model\_id": "dslim\_\_bert-base-ner", "target\_field": "ml.ner\_about\_me", "field\_map": { "aboutme": "text\_field", "ignore\_missing": "true" } } }, { "script": { "lang": "painless", "if": "return ctx['ml']['ner\_resume\_content'].containsKey('entities') && ctx['ml']['ner\_about\_me'].containsKey('entities')", "source": "Map resumeContentTags = new HashMap(); for (item in ctx['ml']['ner\_resume\_content']['entities']) { if (!resumeContentTags.containsKey(item.class\_name)) resumeContentTags[item.class\_name] = new HashSet(); resumeContentTags[item.class\_name].add(item.entity); } ctx['resume\_content\_tags'] = resumeContentTags; Map aboutMeTags = new HashMap(); for (item in ctx['ml']['ner\_about\_me']['entities']) { if (!aboutMeTags.containsKey(item.class\_name)) aboutMeTags[item.class\_name] = new HashSet(); aboutMeTags[item.class\_name].add(item.entity); } ctx['about\_me\_tags'] = aboutMeTags;" } } ], "on\_failure": [ { "set": { "description": "Index document to 'failed-'", "field": "\_index", "value": "failed-{{{ \_index }}}" } }, { "set": { "description": "Set error message", "field": "ingest.failure", "value": "{{\_ingest.on\_failure\_message}}" } } ] } } I should run the above elser or ner pipelines only when **resumecontent** or **aboutme** fields are available. Is there a way we can specify a condition to run the pipeline if one of the fields are available. Can you please help me with how the code looks like with condition? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [November 3, 2023, 9:33pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405/4 "2023-11-03T21:33:29Z")
> [@Manasa4](#): > > As per the documentation I can see it for set and drop processors. The documentation explains how to conditionally run a processor and as explained it applies to every processor. > **Each processor supports** an optional `if` condition, written as a [Painless script](https://www.elastic.co/guide/en/elasticsearch/painless/8.10/painless-guide.html). If provided, the processor only runs when the `if` condition is `true` . If you check the [inference processor documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-processor.html#inference-processor) you will see that the `if` is also an option and has a link to the documentation. > [@Manasa4](#): > > I should run the above elser or ner pipelines only when **resumecontent** or **aboutme** fields are available. > Is there a way we can specify a condition to run the pipeline if one of the fields are available. Can you please help me with how the code looks like with condition? You need to follow the examples in the documentation, there are some that validates if a field exists or not. It will be something like this: ```auto "if": "ctx.resumecontent != null || ctx.aboutme != null" ``` ---
**Author:** ![Manasa4](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@Manasa4](https://discuss.elastic.co/u/Manasa4)\ **Post date:** [November 6, 2023, 3:44pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405/5 "2023-11-06T15:44:20Z")
Thanks for the above condition. I have tried to add my condition like this with "ignore\_missing" : true and also an if condition like below : { "processors": [ { "inference": { "field\_map": { "resumecontent": "text\_field" //"ignore\_missing": "true" }, "if": "ctx.resumecontent != null || ctx.aboutme != null", "model\_id": "dslim\_\_bert-base-ner", "target\_field": "ml.ner\_resume\_content" ``` } }, { "inference": { //"if": "ctx.resumecontent != null || ctx.aboutme != null", "field_map": { "aboutme": "text_field" //"ignore_missing": "true" }, "if": "ctx.resumecontent != null || ctx.aboutme != null", "model_id": "dslim__bert-base-ner", "target_field": "ml.ner_about_me" } }, { "script": { "lang": "painless", "if": "return ctx['ml']['ner_resume_content'].containsKey('entities') && ctx['ml']['ner_about_me'].containsKey('entities')", "source": "Map resumeContentTags = new HashMap(); for (item in ctx['ml']['ner_resume_content']['entities']) { if (!resumeContentTags.containsKey(item.class_name)) resumeContentTags[item.class_name] = new HashSet(); resumeContentTags[item.class_name].add(item.entity); } ctx['resume_content_tags'] = resumeContentTags; Map aboutMeTags = new HashMap(); for (item in ctx['ml']['ner_about_me']['entities']) { if (!aboutMeTags.containsKey(item.class_name)) aboutMeTags[item.class_name] = new HashSet(); aboutMeTags[item.class_name].add(item.entity); } ctx['about_me_tags'] = aboutMeTags;" } } ], "on_failure": [ { "set": { "description": "Index document to 'failed-'", "field": "_index", "value": "failed-{{{ _index }}}" } }, { "set": { "description": "Set error message", "field": "ingest.failure", "value": "{{_ingest.on_failure_message}}" } } ] ``` } I have been trying to run the above pipeline with reindexing which has only **aboutme** field and I don't see anything that is being created on the final index. Can you please let me know if I'm missing on something? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 4, 2023, 3:45pm UTC](https://discuss.elastic.co/t/multiple-pipelines-with-condition/346405/6 "2023-12-04T15:45:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.