# Copy a value from one field to other if a field exists by using ingestnode pipeline

**URL:** <https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [November 17, 2021, 5:18pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484 "2021-11-17T17:18:19Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![sidharth\_vijayakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidharth_vijayakumar/32/97257_2.png) [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\
**Post date:** [November 17, 2021, 5:18pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/1 "2021-11-17T17:18:20Z")

</div>

Hi Team,  
I want to create a new field called kubernetes.pod.name if fields called prometheus.labels.pod exists in the logs. I found out that from the set processor I could copy the value which is present in prometheus.labels.pod to a new field kubernetes.pod.name but I need to do this conditionally as the pod name keeps on changing.

How do i set a condition such that if field prometheus.labels.pod exists then only I need to add a new field called kubernetes.pod.name (both has the same value)

ctx.prometheus?.labels?.namespace== "name\_of\_namespace"

could be do similarly can we do

ctx.prometheus?.labels?.pod== "\*"

to check if this field exists or not ?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [November 17, 2021, 5:35pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/3 "2021-11-17T17:35:56Z")

</div>

See if this works for you.

```auto
POST /_ingest/pipeline/_simulate?verbose=true
{
  "pipeline" :
  {
    "processors": [
      {
        "set" : {
          "field" : "kubernetes.pod.name",
          "value" : "{{prometheus.labels.pod}}",
          "if" : "ctx.containsKey('prometheus.labels.pod')"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "prometheus.labels.pod": "test"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![sidharth\_vijayakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidharth_vijayakumar/32/97257_2.png) [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\
**Post date:** [November 18, 2021, 9:14am UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/4 "2021-11-18T09:14:50Z")

</div>

Hi @aaron-nimocks ,

Nop not working. What's your view on using this in set processor ?

ctx?.prometheus?.labels?.pod!=null

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [November 18, 2021, 12:31pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/5 "2021-11-18T12:31:49Z")

</div>

What exactly does your data look like? The above example works if it's formatted like this.

```auto
      "_source": {
        "prometheus.labels.pod": "test"
      }

```

I don't think `ctx?.prometheus?.labels?.pod!=null` will work. The `ctx.containsKey('prometheus.labels.pod')` checks to see if the field exists and I think that's what you are looking for.

---

<div class="post-metadata">

**Author:** ![sidharth\_vijayakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidharth_vijayakumar/32/97257_2.png) [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\
**Post date:** [November 18, 2021, 12:43pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/6 "2021-11-18T12:43:28Z")

</div>

When i use containsKey it's skipping that set condition. Data looks like this :

"prometheus.labels.pod": [  
"test"  
]

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [November 18, 2021, 1:29pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/7 "2021-11-18T13:29:19Z")

</div>

I don't know the answer yet, might require another processor. But the issue is you have an array based on your last message. I thought it was a string before.

So if you run the below in dev tools you will see one of the 2 messages will be true on the condition and it will create a new field but I am not sure you can target the first array element. Is there always only one value in the array? If there is 2 values then which do you use?

```auto
"kubernetes" : {
 "pod" : {
  "name" : "{0=test}"
 }
}

```

```auto
POST /_ingest/pipeline/_simulate?verbose=true
{
  "pipeline": {
    "processors": [
      {
        "set": {
          "field": "kubernetes.pod.name",
          "value": "{{prometheus.labels.pod}}",
          "if": "ctx.containsKey('prometheus.labels.pod')"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "prometheus.labels.pod": [
          "test"
        ]
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![sidharth\_vijayakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidharth_vijayakumar/32/97257_2.png) [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\
**Post date:** [November 18, 2021, 1:44pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/8 "2021-11-18T13:44:59Z")

</div>

There will be always 1 value and if that field exists I have to create another field kubernetes.pod.name. It won't have more than 1 value.

Any idea why the below would be incorrect ?

ctx?.prometheus?.labels?.pod!=null

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [November 18, 2021, 1:49pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/9 "2021-11-18T13:49:18Z")

</div>

> [@sidharth\_vijayakumar](#):
>
> Any idea why the below would be incorrect ?
> 
> ctx?.prometheus?.labels?.pod!=null

Because it's an array. There is a big difference between.

```auto
"_source": {
 "prometheus.labels.pod": [
  "test"
 ]
}

```

and

```auto
"_source": {
 "prometheus.labels.pod": "test"
}

```

Also if you were just checking to see if the field was null or empty then records without that field would return true and it would create a new field with no value.

---

<div class="post-metadata">

**Author:** ![sidharth\_vijayakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidharth_vijayakumar/32/97257_2.png) [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\
**Post date:** [November 18, 2021, 2:05pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/10 "2021-11-18T14:05:31Z")

</div>

I am so sorry it's a string i just check again kibana says this field is string and I checked my data in json format. then could see

```
"_source": {
"prometheus": {
  "query": {
    "count": 142
  },
  "labels": {
    "pod": "test"
   
   }
 }
}
```

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [November 18, 2021, 2:26pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/11 "2021-11-18T14:26:42Z")

</div>

On that note what you said should work.

```auto
POST /_ingest/pipeline/_simulate?verbose=true
{
  "pipeline": {
    "processors": [
      {
        "set": {
          "field": "kubernetes.pod.name",
          "value": "{{prometheus.labels.pod}}",
          "if": "ctx?.prometheus?.labels?.pod != null"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "prometheus": {
          "query": {
            "count": 142
          },
          "labels": {
            "pod": "test"
          }
        }
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![sidharth\_vijayakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidharth_vijayakumar/32/97257_2.png) [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\
**Post date:** [November 18, 2021, 4:39pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/12 "2021-11-18T16:39:28Z")

</div>

Yes, I tested for different cases and works as expected. Thanks a lot for your time and analysis. Much appreciated!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 16, 2021, 4:39pm UTC](https://discuss.elastic.co/t/copy-a-value-from-one-field-to-other-if-a-field-exists-by-using-ingestnode-pipeline/289484/13 "2021-12-16T16:39:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.