# Ingest Pipeline, apply only if fields exist

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564>\
**Category:** Elasticsearch\
**Created:** [March 22, 2019, 10:46pm UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564 "2019-03-22T22:46:00Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 22, 2019, 10:46pm UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564/1 "2019-03-22T22:46:00Z")

</div>

Hello,

I have a ingest pipeline that does two things. It basically leverages kv and it applies geo info to a field.

```
PUT _ingest/pipeline/mitresplit
{
    "description": "splits technique_name and technique_id",
    "processors": [
      {
        "kv": {
          "field": "sfRuleName",
          "field_split": ",",
          "value_split": "="
        },
        "geoip": {
          "field": "sfDestinationIp",
          "target_field": "geo"
        }
      }
    ]
  }

```

And apply the pipeline

```
 PUT winlogbeat-*/_settings
 {
"index.default_pipeline":"_default"
 }

```

What I have figured out is if the document is sent in does NOT contain one of the field names, then it just dies.

How can I skin this cat? If either of the fields are missing, I want things to continue to work.

---

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 22, 2019, 11:07pm UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564/2 "2019-03-22T23:07:11Z")

</div>

I think that I have figured it out by leveraging the below:

```
PUT _ingest/pipeline/mitresplit
{
"description": "splits technique_name and technique_id",
"processors": [
  {
    "kv": {
      "field": "event_data.RuleName",
      "field_split": ",",
      "value_split": "=",
      "ignore_failure" : true
    },
    "geoip": {
      "field": "event_data.DestinationIp",
      "ignore_failure" : true,
      "target_field": "geo"
    }
  }
]
}

PUT winlogbeat-*/_settings
{
"index.default_pipeline":"mitresplit"
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 23, 2019, 5:52am UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564/3 "2019-03-23T05:52:28Z")

</div>

Which version of Elasticsearch are you using?

---

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 23, 2019, 11:55am UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564/4 "2019-03-23T11:55:48Z")

</div>

6.6.0.

BTW, the

> "ignore\_failure" : true

Seems to have fixed the issue, but I don't like failures. It would be better (I think) to have a conditional statement that says, if field exists then do.

Thanks for the help.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 23, 2019, 2:19pm UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564/5 "2019-03-23T14:19:43Z")

</div>

Can you not use a [conditional](https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest-conditionals.html)?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 20, 2019, 2:19pm UTC](https://discuss.elastic.co/t/ingest-pipeline-apply-only-if-fields-exist/173564/6 "2019-04-20T14:19:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.