Is it possible to include regex in dissect filter?

Elastic Stack Logstash
1

Hi,

I am using ELK GA 6.3.0. I am using Logstash to read data from Kafka. I have messages like;

 1  0 1707820 142124 198704 511288    0    0     0   144

There is a white space at the beginning of message. There are spaces in between, acting as delimiters. The problem is that the number of white space might vary. Sometimes 1, sometimes 2 etc. The number of attributes is constant (10). I was using grok filter to split the message like;

grok{
	match => { "message" => "%{NUMBER:data1}\s*%{NUMBER:data2}\s*%{NUMBER:data3}\s*%{NUMBER:data4}\s*%{NUMBER:data5}\s*%{NUMBER:data6}\s*%{NUMBER:data7}\s*%{NUMBER:data8}\s*%{NUMBER:data9}\s*%{NUMBER:data10}" }
}

I would like to see if this is possible with dissect filter. I have tried;

dissect{
	mapping => { "message" => "%{data1}\s*%{data2}\s*%{data3}\s*%{data4}\s*%{data5}\s*%{data6}\s*%{data7}\s*%{data8}\s*%{data9}\s*%{data10}" }
}

But this is not working. Is this possible?

Thanks.

2

I do not think the number of spaces can vary.
Personally, I often use : http://grokconstructor.appspot.com/

3

You need to use the -> notation to handle varying number of separators. See the docs for an example of the syntax.

4

Ok, I understand, (and I had already gone through that :wink:) But what if my data is like 0\t 2?

Tried %{data1}\t %{data2} and it gave data1 "" and data2 "0\t 2".

Tried %{data1} %{data2} and it gave data1 "0\t" and data2 "2".

Is there any way to fix this?

5

Then you might be better off using grok with the SPACE pattern.