Is it possible to include regex in dissect filter?


I am using ELK GA 6.3.0. I am using Logstash to read data from Kafka. I have messages like;

 1  0 1707820 142124 198704 511288    0    0     0   144

There is a white space at the beginning of message. There are spaces in between, acting as delimiters. The problem is that the number of white space might vary. Sometimes 1, sometimes 2 etc. The number of attributes is constant (10). I was using grok filter to split the message like;

	match => { "message" => "%{NUMBER:data1}\s*%{NUMBER:data2}\s*%{NUMBER:data3}\s*%{NUMBER:data4}\s*%{NUMBER:data5}\s*%{NUMBER:data6}\s*%{NUMBER:data7}\s*%{NUMBER:data8}\s*%{NUMBER:data9}\s*%{NUMBER:data10}" }

I would like to see if this is possible with dissect filter. I have tried;

	mapping => { "message" => "%{data1}\s*%{data2}\s*%{data3}\s*%{data4}\s*%{data5}\s*%{data6}\s*%{data7}\s*%{data8}\s*%{data9}\s*%{data10}" }

But this is not working. Is this possible?


I do not think the number of spaces can vary.
Personally, I often use :

You need to use the -> notation to handle varying number of separators. See the docs for an example of the syntax.


Ok, I understand, (and I had already gone through that :wink:) But what if my data is like 0\t 2?

Tried %{data1}\t %{data2} and it gave data1 "" and data2 "0\t 2".

Tried %{data1} %{data2} and it gave data1 "0\t" and data2 "2".

Is there any way to fix this?

Then you might be better off using grok with the SPACE pattern.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.