Hi I am importing data from mssql database into Elasticsearch. I am trying to split field "my_message" that is structured as follows: "{1:Mokete}{2:Mokoena}{3:{888:Logstash}}{4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:0000000…

I'd recommend a grok or a dissect filter for this problem.

Hi @magnusbaeck Thanks for the quick reply. Attempted grok, dissect and kv filter but couldn't get the syntax correct as they we all throwing errors. Pipeline aborted due to error {:exception=>"Grok::PatternError", :error=>"pattern %{my_message} not defined", :backtrace=>["/opt/logstash/vendor/bu…

@magnusbaeck I used the below : filter { grok { match => { "message" => "%{my_message}" } } }

That looks like dissect syntax in a grok filter. Switch to a dissect filter or use the grok constructor web site to construct a grok expression that you can use.

@magnusbaeck swithed to disssect filter and getting the following fetched an invalid config {:config=>************* :reason=>"Couldn't find any filter plugin named 'dissect'. Are you sure this is correct? Trying to load the dissect filter plugin resulted in this error: no such file to load -- logs…

on grok website provided input: {1:Name} {2:Surname} {3:{888:Logstash}} {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}} pattern : (?{\d:.}) (?{\d:.})(?{\d:.*}) results: { "message": [ [ "{1:Nam…

2.4 is probably too old for dissect. You might be able to install the plugin via logstash-plugin install but otherwise you have to upgrade Logstash.

Discuss the Elastic Stack

Logstash split fields into new fields(field1, field2, field3, field4, field5)

Elastic Stack Logstash

magnusbaeck (Magnus Bäck) February 21, 2018, 1:35pm 4

And what configuration gave you the error above?

Topic		Replies	Views
Split Logfiles Logstash	4	263	July 26, 2018
How to use split filter on the field using logstash Logstash	5	6372	March 27, 2019
[SOLVED] Grok and split fields Logstash	5	8836	July 6, 2017
How to Split the one field into multiple fields using kv plugins Logstash	5	2715	April 11, 2019
Create fields by splitting a log Logstash	8	1277	April 2, 2020

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Logstash split fields into new fields(field1, field2, field3, field4, field5)

Related topics