# Split source value and create a custom field with splitted one **URL:** \ **Category:** Logstash\ **Created:** [December 5, 2017, 12:02pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334 "2017-12-05T12:02:10Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![platso588](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@platso588](https://discuss.elastic.co/u/platso588)\ **Post date:** [December 5, 2017, 12:02pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334/1 "2017-12-05T12:02:10Z")
Hi All, I have merged few Yarn logs in elasticsearch and in Kibana. source=/data06/yarn/logs/application\_1510757389739\_0634/container\_e15\_1510757554319\_0034\_01\_000001/hive2kafka-export29.log From source, I want to extract 'application\_1510757389739\_0634' and add it to a new field applicationID. I have created below filter in logstash: if ([fields][log\_type] == "yarnHive2kafkaLog") { grok { match =\> { "message" =\> "%{YEAR:logYear}-%{MONTHNUM:logMonth}-%{MONTHDAY:logDate} %{TIME:logTime} !%{SPACE}%{LOGLEVEL:logLevel}%{SPACE}! %{GREEDYDATA:message}"} } mutate { split =\> ["source", "/"] add\_field =\> { "applicationID" =\> "%{source[4]}" } } } I could extract 'application\_1510757389739\_0634', however **source** field went for a toss and became a CSV value, , data06, yarn, logs, application\_1510757389739\_0634, container\_e15\_1510757327319\_0034\_01\_000001, hive2kafka-export30.log Below Screenshot shows this. ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/7/47e2c29951e75125a4198aa7b76dde25bfc62c7a.png) How do I tackle with this? Thanks in Advance. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 5, 2017, 12:11pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334/2 "2017-12-05T12:11:17Z")
You asked the mutate filter to split `source` into an array and that's what it did. There are many ways to solve this, including making a copy of `source` before you split it or using a grok filter to extract the correct directory component. ---
**Author:** ![platso588](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@platso588](https://discuss.elastic.co/u/platso588)\ **Post date:** [December 5, 2017, 1:01pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334/3 "2017-12-05T13:01:12Z")
Thanks Magnus. I tried the suggested 2 ways: **1. copy of source by creating a temp variable as below,** if ([fields][log\_type] == "yarnHive2kafkaLog") { grok { match =\> { "message" =\> "%{YEAR:logYear}-%{MONTHNUM:logMonth}-%{MONTHDAY:logDate} %{TIME:logTime} !%{SPACE}%{LOGLEVEL:logLevel}%{SPACE}! %{GREEDYDATA:message}"} } mutate { copy =\> { "source" =\> "source\_tmp" } split =\> ["source\_tmp", "/"] add\_field =\> { "applicationID" =\> "%{source\_tmp[3]}" } } } I could not get EXTRACTED applicationID. Output is as below: ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/a/9adcad3e28688ee1ccfe99868ef6daab68b69bbc.png) **2. grok filter on source** snippet: if ([fields][log\_type] == "yarnHive2kafkaLog") { grok { match =\> { "message" =\> "%{YEAR:logYear}-%{MONTHNUM:logMonth}-%{MONTHDAY:logDate} %{TIME:logTime} !%{SPACE}%{LOGLEVEL:logLevel}%{SPACE}! %{GREEDYDATA:message}"} match =\> { "source" =\> "/%{GREEDYDATA:primaryDir}/%{GREEDYDATA:subDir1}/%{GREEDYDATA:subDir2}/%{GREEDYDATA:subDir3}/%{GREEDYDATA:containerID}/%{GREEDYDATA:fileName}"} } mutate { add\_field =\> { "applicationID" =\> "%{subDir3}" } } } I have same Issue in this as well applicationID has value as %{subDir3}. Could you please correct me on this? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 5, 2017, 1:15pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334/4 "2017-12-05T13:15:54Z")
> mutate { > copy =\> { "source" =\> "source\_tmp" } > split =\> ["source\_tmp", "/"] > add\_field =\> { "applicationID" =\> "%{source\_tmp[3]}" } > } `split` applies before `copy`. You need to use at least two mutate filters after one another. Similar thing with the grok filter solution. The grok filter breaks after the first match so you need to match against `source` in a separate grok filter. ---
**Author:** ![platso588](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@platso588](https://discuss.elastic.co/u/platso588)\ **Post date:** [December 5, 2017, 1:37pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334/5 "2017-12-05T13:37:54Z")
This resolved the Issue and Thanks a ton 👌 Working code snippets for both the approaches. **1. copy of source by creating a temp variable,** ``` if ([fields][log_type] == "yarnHive2kafkaLog") { grok { match => { "message" => "%{YEAR:logYear}-%{MONTHNUM:logMonth}-%{MONTHDAY:logDate} %{TIME:logTime} \!%{SPACE}%{LOGLEVEL:logLevel}%{SPACE}\! %{GREEDYDATA:message}"} } mutate { copy => { "source" => "source_tmp" } } mutate { split => ["source_tmp", "/"] add_field => { "applicationID" => "%{source_tmp[4]}" } } } ``` **2. grok filter on source** ``` if ([fields][log_type] == "yarnHive2kafkaLog") { grok { match => { "message" => "%{YEAR:logYear}-%{MONTHNUM:logMonth}-%{MONTHDAY:logDate} %{TIME:logTime} \!%{SPACE}%{LOGLEVEL:logLevel}%{SPACE}\! %{GREEDYDATA:message}"} } grok { match => { "source" => "/%{GREEDYDATA:primaryDir}/%{GREEDYDATA:subDir1}/%{GREEDYDATA:subDir2}/%{GREEDYDATA:subDir3}/%{GREEDYDATA:containerID}/%{GREEDYDATA:fileName}"} } mutate { add_field => { "applicationID" => "%{subDir3}" } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 2, 2018, 1:38pm UTC](https://discuss.elastic.co/t/split-source-value-and-create-a-custom-field-with-splitted-one/110334/6 "2018-01-02T13:38:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.