Dissect in logstash and tabs

[astateofmind (Laurentiu Vasiescu)]

Trying to use dissect to add log.level field to some beats.

Using filebeat to send the data and some logs have their fields separated by tabs instead of spaces.

The logs with space work ok with this filter:

"%{} %{log.level->} %{}"

If I try something similar:

"%{}\t%{log.level->}\t[%{}"

I get the warn:

pattern"=>"%{}\t%{log.level->}\t%{} = not found.

Weird that if I try that in https://dissect-tester.jorgelbg.me it seems to work ok ...

Any ideas why it does that to \t and what to use instead?

[leandrojmp (Leandro Pereira)]

Can you share some sample logs and the configuration you are using in Logstash?

Also, is it a typo? The patterns you shared are different, one has a [%{} and the other does not have it.

[astateofmind (Laurentiu Vasiescu)]

Yes, it was a typo, corrected.

The error is:

"org.logstash.dissect.Dissector][main][e5c3abef8a6cef340816f3cd3afa186880e34a70a6ed6301de76a9436ec9c59b] Dissector mapping, pattern not found {"field"=>"message", "pattern"=>"%{}\\t%{log.level}\\t%{}"

Ad the pattern was how it's above, apparently is adds and extra \ for some reason.

I did a mutate prior to pattern matching, replacing \t with " " and it works ok now matching the " " so I guess for now I can live with that ...

Still, weird that \t is not accepted.

[Badger]

dissect does not support any form of regexp or character escapes. The reason it is so much faster than grep is that it simply scans forward through the field looking for the next delimiter it expects. So, yes, you need a literal tab in the pattern.