# Logstash split fields into new fields(field1, field2, field3, field4, field5)
**URL:** https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840
**Category:** Logstash
**Created:** [February 21, 2018, 12:54pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840 "2018-02-21T12:54:05Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 21, 2018, 12:54pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/1 "2018-02-21T12:54:05Z")
Hi
I am importing data from mssql database into Elasticsearch.
I am trying to split field "my\_message" that is structured as follows:
"{1:Mokete}{2:Mokoena}{3:{888:Logstash}}{4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}}"
I would like to split the above field to :
field1: {1:Name}
field2: {2:Surname}
field3: {3:{888:Logstash}}
field4: {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }
field5: {5:{MY:00000000}{ZAR:Testing}{Test:}}
I tried the following filter:
filter {
mutate
{
split =\> { "my\_message" =\> "{?:"}
add\_field =\>
{
"field1" =\> "%{[my\_message][1]}"
"field2" =\> "%{[my\_message][2]}"
"field3" =\> "%{[my\_message][3]}"
"field4" =\> "%{[my\_message][4]}"
"field5" =\> "%{[my\_message][5]}"
}
}
}
this gives me the following results:
field1: {1:Name} {2:Surname} {3:{888:Logstash}} {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}}
Expectation:
to go throught "my\_message" field and where it finds a "{1:", "{2:","{3:","{4:","{5:" split the message and return :
field1: Name}
field2: Surname}
field3: {888:Logstash}}
field4: \n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }
field5: {MY:00000000}{ZAR:Testing}{Test:}}
Thanks,
Mokete
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 21, 2018, 1:14pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/2 "2018-02-21T13:14:14Z")
I'd recommend a grok or a dissect filter for this problem.
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 21, 2018, 1:22pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/3 "2018-02-21T13:22:50Z")
Hi @magnusbaeck
Thanks for the quick reply.
Attempted grok, dissect and kv filter but couldn't get the syntax correct as they we all throwing errors.
Pipeline aborted due to error {:exception=\>"Grok::PatternError", :error=\>"pattern %{my\_message} not defined", :backtrace=\>["/opt/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:123:in `compile'", "org/jruby/RubyKernel.java:1479:in`loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:93:in `compile'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-2.0.5/lib/logstash/filters/grok.rb:264:in`register'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-2.0.5/lib/logstash/filters/grok.rb:259:in`register'", "org/jruby/RubyHash.java:1342:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-2.0.5/lib/logstash/filters/grok.rb:255:in`register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:182:in `start_workers'", "org/jruby/RubyArray.java:1613:in`each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:182:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:136:in`run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/agent.rb:491:in `start\_pipeline'"], :level=\>:error}
Please advice on the syntax for both grok and dissect.
Thanks
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 21, 2018, 1:35pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/4 "2018-02-21T13:35:28Z")
And what configuration gave you the error above?
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 21, 2018, 1:39pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/5 "2018-02-21T13:39:57Z")
@magnusbaeck
I used the below :
filter {
grok {
match =\> { "message" =\> "%{my\_message}" }
}
}
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 21, 2018, 2:09pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/6 "2018-02-21T14:09:10Z")
That looks like dissect syntax in a grok filter. Switch to a dissect filter or use the grok constructor web site to construct a grok expression that you can use.
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 21, 2018, 2:18pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/7 "2018-02-21T14:18:47Z")
@magnusbaeck
swithed to disssect filter and getting the following
fetched an invalid config {:config=\>\*\*\*\*\*\*\*\*\*\*\*\*\* :reason=\>"Couldn't find any filter plugin named 'dissect'. Are you sure this is correct? Trying to load the dissect filter plugin resulted in this error: no such file to load -- logstash/filters/dissect", :level=\>:error}
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 21, 2018, 2:20pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/8 "2018-02-21T14:20:24Z")
What version of Logstash?
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 21, 2018, 2:21pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/9 "2018-02-21T14:21:04Z")
on grok website
provided input:
{1:Name} {2:Surname} {3:{888:Logstash}} {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}}
pattern :
(?{\d:._}) (?{\d:._})(?{\d:.\*})
results:
{
"message": [
[
"{1:Name} {2:Surname} {3:{888:Logstash}}"
]
]
}
pattern :
(?{\d:._}) (?{\d:._})(?{\d:._})(?{\d:._})
results:
No Matches
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 21, 2018, 2:21pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/10 "2018-02-21T14:21:38Z")
@magnusbaeck
version 2.4
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 21, 2018, 2:27pm UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/11 "2018-02-21T14:27:39Z")
2.4 is probably too old for dissect. You might be able to install the plugin via `logstash-plugin install` but otherwise you have to upgrade Logstash.
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 22, 2018, 8:49am UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/12 "2018-02-22T08:49:08Z")
@magnusbaeck, i managed to construct grok syntax from grok constructor web site but when i run Logstash i get the following output.
"\_index": "messages",
"\_type": "message",
"\_id": "1111",
"\_score": 1,
"\_source": {
"my\_message": "{1:Name} {2:Surname} {3:{888:Logstash}} {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}}",
"tags": [
"\_grokparsefailure"
]
}
}
Thanks
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 22, 2018, 8:52am UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/13 "2018-02-22T08:52:26Z")
To debug, start with the simplest possible grok expression. Does it work? If yes, continue by adding more and more and it breaks. That narrows down the problem.
---
### Author:  [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)
#### Post date: [February 22, 2018, 8:56am UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/14 "2018-02-22T08:56:45Z")
> [@magnusbaeck](#):
>
> To debug, start with the simplest possible grok expression. Does it work? If yes, continue by adding more and more and it breaks. That narrows down the problem.
This is my expression:
filter {
grok {
match =\> { "%{my\_message}" =\> "(?{1:.\*?})" }
}
}
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 22, 2018, 9:05am UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/15 "2018-02-22T09:05:18Z")
Replace `%{fin_format}` with `message` since that's the field you want to parse.
Always format configuration snippets as preformatted text so the posting software doesn't mangle it.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 22, 2018, 9:14am UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840/17 "2018-03-22T09:14:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.