# Access Field in Ingest-Pipeline

**URL:** <https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367>\
**Category:** Elasticsearch\
**Created:** [May 2, 2019, 12:56pm UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367 "2019-05-02T12:56:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![nimda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nimda/32/44386_2.png) [@nimda](https://discuss.elastic.co/u/nimda)\
**Post date:** [May 2, 2019, 12:56pm UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/1 "2019-05-02T12:56:41Z")

</div>

Hello Everyone,

I'm currently trying out Ingest-Pipelines, but I'm stuck because of some "field access" issues. My Pipeline is the following.

> POST \_ingest/pipeline/\_simulate  
> {  
> "pipeline" : {  
> "description" : "pipeline for splitting hashes stored in the event.Hash field",  
> "processors" : [  
> {  
> "kv" : {  
> "field": "event\_data.Hash",  
> "field\_split": ",",  
> "value\_split": "=",  
> "target\_field": "event\_data",  
> "ignore\_missing": true  
> },  
> "set": {  
> "if": "event\_data.SHA256 == 'A993F8C574E0FEA8C1CDCBCD9408D9E2E107EE6E4D120EDCFA11DECD53FA0CAE'",  
> "field": "suspicious",  
> "value": "true"  
> },  
> "remove": {  
> "field": "event\_data.Hash",  
> "ignore\_missing": true  
> }  
> }  
> ]  
> },  
> "docs": [  
> {  
> "\_source" : {  
> "event\_data" : {  
> "Hash" : "SHA1=9FA11A63B43F83980E0B48DC9BA2CB59D545A4E8,MD5=D7B20F933BE6CDAE41EFBE75548EBA5F,SHA256=A993F8C574E0FEA8C1CDCBCD9408D9E2E107EE6E4D120EDCFA11DECD53FA0CAE,IMPHASH=D989D7ADF6957F1A88BB1332E40317E6"  
> }  
> }  
> }  
> ]  
> }

Unfortunatly my document isn't marked as "suspicious". I tried using the set-processor "inline" like mentioned here and putting it into a different pipeline. Both didn't work.

---

<div class="post-metadata">

**Author:** ![nimda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nimda/32/44386_2.png) [@nimda](https://discuss.elastic.co/u/nimda)\
**Post date:** [May 6, 2019, 8:02am UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/2 "2019-05-06T08:02:41Z")

</div>

Any ideas, why the pipeline isn't working as intended?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [May 6, 2019, 8:07am UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/3 "2019-05-06T08:07:24Z")

</div>

I just tested this on 7.0 and if I start the `if` part in the `set` processor with `"ctx.event_data.SHA256 ...` it works for me[tm].

What version are you on?

--Alex

---

<div class="post-metadata">

**Author:** ![nimda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nimda/32/44386_2.png) [@nimda](https://discuss.elastic.co/u/nimda)\
**Post date:** [May 6, 2019, 8:11am UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/4 "2019-05-06T08:11:25Z")

</div>

Thanks for the response. I'm using Elasticsearch 7.0.0 and unfortunatly the document isn't marked as "supicious". This is my response if I run the pipeline:

> {  
> "docs" : [  
> {  
> "doc" : {  
> "\_index" : "\_index",  
> "\_type" : "\_doc",  
> "\_id" : "\_id",  
> "\_source" : {  
> "event\_data" : {  
> "SHA256" : "A993F8C574E0FEA8C1CDCBCD9408D9E2E107EE6E4D120EDCFA11DECD53FA0CAE",  
> "SHA1" : "9FA11A63B43F83980E0B48DC9BA2CB59D545A4E8",  
> "MD5" : "D7B20F933BE6CDAE41EFBE75548EBA5F",  
> "IMPHASH" : "D989D7ADF6957F1A88BB1332E40317E6"  
> }  
> },  
> "\_ingest" : {  
> "timestamp" : "2019-05-06T08:09:44.740576Z"  
> }  
> }  
> }  
> ]  
> }

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [May 6, 2019, 8:30am UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/5 "2019-05-06T08:30:12Z")

</div>

Please take your time to properly format and indent your messages. Because this is really the issue here. The `set` processor is within the `script` processor.

Check this example

```auto
POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "pipeline for splitting hashes stored in the event.Hash field",
    "processors": [
      {
        "kv": {
          "field": "hash",
          "field_split": ",",
          "value_split": "=",
          "target_field": "output",
          "tag": "kv"
        }
      },
      {
        "set": {
          "if": "ctx.output.SHA256 == '1'",
          "field": "suspicious",
          "value": "true",
          "tag": "set"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "hash": "SHA1=2,MD5=3,SHA256=1,IMPHASH=4"
      }
    }
  ]
}

```

The `set` processor now is its own element with in the `processors` array.

Nontheless, an error message might have been useful here. I will open an issue.

---

<div class="post-metadata">

**Author:** ![nimda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nimda/32/44386_2.png) [@nimda](https://discuss.elastic.co/u/nimda)\
**Post date:** [May 6, 2019, 8:43am UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/6 "2019-05-06T08:43:34Z")

</div>

Ahhh I see the problem now. I just overlooked the fact that I had to put each processor in it's own curly brackets, even tho it is clearly stated in the documentation.

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 3, 2019, 8:43am UTC](https://discuss.elastic.co/t/access-field-in-ingest-pipeline/179367/7 "2019-06-03T08:43:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.