I am importing data from mssql database into Elasticsearch.
I am trying to split field "my_message" that is structured as follows:
"{1:Mokete}{2:Mokoena}{3:{888:Logstash}}{4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}}"
I would like to split the above field to :
field1: {1:Name}
field2: {2:Surname}
field3: {3:{888:Logstash}}
field4: {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }
field5: {5:{MY:00000000}{ZAR:Testing}{Test:}}
this gives me the following results:
field1: {1:Name} {2:Surname} {3:{888:Logstash}} {4:\n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }{5:{MY:00000000}{ZAR:Testing}{Test:}}
Expectation:
to go throught "my_message" field and where it finds a "{1:", "{2:","{3:","{4:","{5:" split the message and return :
field1: Name}
field2: Surname}
field3: {888:Logstash}}
field4: \n:20:Testing\n:LG:Logstash\n:ES:Elasticsearch,\n:CURR:ZAR,\n:HM:/Home\nHOME Jake\n111 FIRST ROAD \n ZA }
field5: {MY:00000000}{ZAR:Testing}{Test:}}
That looks like dissect syntax in a grok filter. Switch to a dissect filter or use the grok constructor web site to construct a grok expression that you can use.
swithed to disssect filter and getting the following
fetched an invalid config {:config=>************* :reason=>"Couldn't find any filter plugin named 'dissect'. Are you sure this is correct? Trying to load the dissect filter plugin resulted in this error: no such file to load -- logstash/filters/dissect", :level=>:error}
To debug, start with the simplest possible grok expression. Does it work? If yes, continue by adding more and more and it breaks. That narrows down the problem.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.