# Ingest pipeline KV processor failure

**URL:** https://discuss.elastic.co/t/ingest-pipeline-kv-processor-failure/94691
**Category:** Elasticsearch
**Created:** [July 26, 2017, 7:10pm UTC](https://discuss.elastic.co/t/ingest-pipeline-kv-processor-failure/94691 "2017-07-26T19:10:06Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![tarp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarp/32/51884_2.png) [@tarp](https://discuss.elastic.co/u/tarp)
#### Post date: [July 26, 2017, 7:10pm UTC](https://discuss.elastic.co/t/ingest-pipeline-kv-processor-failure/94691/1 "2017-07-26T19:10:06Z")

</div>

Hi,  
I'm running ES 5.4.1, I'm sending a file to my ingest nodes that looks like this.

time:1500651652886|serial:RWGSIPA530083|appName:DataSyncTab|data:This is a string log message  
time:1500651652887|serial:RWGSIPA530083|appName:DataSyncTab|page:Opened DataSync Diagnostic Page|status:MTG connection active

My ingest pipeline is this,

PUT \_ingest/pipeline/gen4analytics  
{  
"description": "gen4analytics",  
"processors": [  
{  
"kv": {  
"field": "message",  
"field\_split": "|",  
"value\_split": ":"  
}  
},  
{  
"date": {  
"field": "time",  
"formats": ["UNIX\_MS"]  
}  
},  
{  
"date\_index\_name": {  
"field": "@timestamp",  
"index\_name\_format": "yyyy.MM.dd",  
"index\_name\_prefix": "gen4analytics-",  
"date\_rounding": "d"  
}  
}  
],  
"on\_failure": [  
{  
"set": {  
"field": "\_index",  
"value": "failed-{{ \_type }}"  
}  
},  
{  
"set": {  
"field": "error",  
"value": "{{ \_ingest.on\_failure\_message }}"  
}  
}  
]  
}

I'm getting a failure that "field [message] does not contain value\_split [:]" .  
I don't get it because my field clearly contains ":".  
Thanks,  
Tim

---

<div class="post-metadata">

### Author: ![talevy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talevy/32/44896_2.png) [@talevy](https://discuss.elastic.co/u/talevy)
#### Post date: [July 28, 2017, 5:58pm UTC](https://discuss.elastic.co/t/ingest-pipeline-kv-processor-failure/94691/2 "2017-07-28T17:58:36Z")

</div>

Since these character splits represent regex expressions. the `|` needs to be escaped  
here is an example that illustrates. without the `\\`, this pipeline runs into the same problem as you mention.

```
POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "test colon",
    "processors": [
      {
        "kv": {
          "field": "message",
          "field_split": "\\|",
          "value_split": ":"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": { "message": "time:1500651652887|serial:RWGSIPA530083" }
    }
  ]
}
```

---

<div class="post-metadata">

### Author: ![tarp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarp/32/51884_2.png) [@tarp](https://discuss.elastic.co/u/tarp)
#### Post date: [August 1, 2017, 6:07pm UTC](https://discuss.elastic.co/t/ingest-pipeline-kv-processor-failure/94691/3 "2017-08-01T18:07:27Z")

</div>

Thank you, that was what I needed.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 29, 2017, 6:07pm UTC](https://discuss.elastic.co/t/ingest-pipeline-kv-processor-failure/94691/4 "2017-08-29T18:07:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
