# Split a field in elastic search

**URL:** <https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030>\
**Category:** Elasticsearch\
**Created:** [March 19, 2019, 5:59pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030 "2019-03-19T17:59:57Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 19, 2019, 5:59pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/1 "2019-03-19T17:59:57Z")

</div>

I have winbeats installed and things are working great. I have a field that is called:  
event\_data.RuleName

and an actual value is:  
technique\_id=T1130,technique\_name=Install Root Certificate

I would like to split that field into two fields.  
One  
technique\_id and the other technique\_name. Each field would have its perspective values.

Would I use something like split field?  
{  
"split": {  
"field": "event\_data.RuleName",  
"separator": ","  
}  
}

If so, how do I apply this to any index starting with winlogbeat-\*

Thanks,

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [March 19, 2019, 7:13pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/2 "2019-03-19T19:13:01Z")

</div>

Hey @Tim_Rice

You can definitely do this, I am thinking you would want the [kv-processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/kv-processor.html)

Example:

```auto
POST _ingest/pipeline/_simulate
{
  "pipeline" : {
    "processors": [
      {"kv": {
          "field": "event_data.RuleName",
          "field_split": ",",
          "value_split": "="
     }}]
  },
  "docs" : [
    { "_source": {"event_data.RuleName":"technique_id=T1130,technique_name=Install Root Certificate"} }
  ]
}

```

This will result in:

```auto
{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_type",
        "_id" : "_id",
        "_source" : {
          "technique_name" : "Install Root Certificate",
          "ruleName" : "technique_id=T1130,technique_name=Install Root Certificate",
          "technique_id" : "T1130"
        },
        "_ingest" : {
          "timestamp" : "2019-03-19T19:04:53.588Z"
        }
      }
    }
  ]
}

```

For creating an ingest pipeline see the [put-pipeline-api](https://www.elastic.co/guide/en/elasticsearch/reference/master/put-pipeline-api.html).

As for applying an IngestPipeline to an index by default, you can set the [index.default\_pipeline](https://www.elastic.co/guide/en/elasticsearch/reference/master/index-modules.html#dynamic-index-settings) setting.

To apply a default setting to indices matching a pattern use [indices-templates](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-templates.html)

I would look to makes sure that there are no existing templates or pipelines for those types of indices already. You may not want to remove good and existing behavior.

---

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 19, 2019, 7:55pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/4 "2019-03-19T19:55:11Z")

</div>

Thanks for the assistance. I tried your suggestion, and nothing worked.

I also tried:  
PUT \_ingest/pipeline/mitresplit  
{  
"description": "splits technique\_name and technique\_id",  
"processors": [  
{  
"kv": {  
"field": "event\_data.RuleName",  
"field\_split": ",",  
"value\_split": "="  
}  
}  
]  
}

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [March 19, 2019, 8:01pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/5 "2019-03-19T20:01:02Z")

</div>

@Tim_Rice, I am unsure what you mean by "nothing worked".

Is the pipeline (the one you created `mitresplit`) associated with the index through the `index.default_pipeline` option? Also, this will only work on new documents being indexed.

---

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 19, 2019, 9:58pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/6 "2019-03-19T21:58:33Z")

</div>

I really don't know how to add the pipeline to the index.default\_pipeline. I am still trying to research online to see if I can figure this out.

In respects to the "nothing worked", I should have been more descriptive there. Sorry about that.  
It appears that if the "field" has a dot in it, I receive a java error. In this example, the field is event\_data.RuleName

If I change this to event\_dataRulename in your simulation, it works just fine.

The error:

> ```
> "reason" : "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: field [event_data] not present as part of path [event_data.RuleName]",
> 
> ```

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [March 20, 2019, 12:30pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/7 "2019-03-20T12:30:57Z")

</div>

Heya @Tim_Rice, my simulated pipeline is indeed wrong :). Looking at how the data is structured, it seems that it is of the form

```auto
{"event_data":{"ruleName":"technique_id=T1130,technique_name=Install Root Certificate"}}

```

So, I wrote a new simulation and it works just fine with data in that format:

```auto
POST _ingest/pipeline/_simulate
{
  "pipeline" : {
    "processors": [
      {
        "kv": {
          "field": "event_data.ruleName",
          "field_split": ",",
          "value_split": "="
        }
      }
    ]
  },
  "docs" : [
    { "_source": {"event_data":{"ruleName":"technique_id=T1130,technique_name=Install Root Certificate"}} }
  ]
}

```

As for updating the ephemeral settings:

```auto
PUT winlogbeat-*/_settings/
{
  "index.default_pipeline":"my-pipeline"
}

```

CAUTION: I am not sure that `winlogbeat` does not already have a `default_pipeline` defined, you should be able to see if there is just by looking at the settings.

To make sure that the pipeline is continually set on new `winlogbeat-*` indices, you can update the existing template (to see templates call `GET _cat/templates`).

---

<div class="post-metadata">

**Author:** ![Tim\_Rice](https://avatars.discourse-cdn.com/v4/letter/t/c77e96/32.png) [@Tim\_Rice](https://discuss.elastic.co/u/Tim_Rice)\
**Post date:** [March 20, 2019, 1:16pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/8 "2019-03-20T13:16:16Z")

</div>

BenTrent, YOU ARE THE MAN!!!! Everything works perfectly!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2019, 1:16pm UTC](https://discuss.elastic.co/t/split-a-field-in-elastic-search/173030/9 "2019-04-17T13:16:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.