Matching yyyy-MM-dd HH:mm:ss.SSSZ in a logstash filter

I'm trying to extract some fields from this log entry:

2021-07-26 16:45:59.4640|0|WARN|LoggerTestASP.Controllers.WeatherForecastController|NEW_FROM_NLOG: 67b9de16-47a1-4308-a163-263f5f06e841, Hola Pola, 11/26/2017 16:45:59 |url: https://localhost/WeatherForecast|action: GetAll|LoggerTestASP.Controllers.WeatherForecastController.GetAll| body:

I only need the first field (timestamp), the second and third fields as custom, and the remaining as a message.

I tried this in logstash filter:

filter {
        grok {
             match => {"message" =>"%{'yyyy-MM-dd HH:mm:ss.SSSZ':tstmp}|%{NUMBER:myevent}|%{WORD:mylevel}*"}

I'm totally new to Elasticsearch, and not familiar with filtering , tried some online example but couldn't find similar format : 'yyyy-MM-dd HH:mm:ss.SSSZ'

The output (in Kibana) only recognizes myevent, and get its value as 2021, my guess is that it failed to extract the datetime , so second filter (myevent) gets the first "NUMBER" as its value

If | is not escaped then it is used for alternation. That means that your pattern matches 'yyyy-MM-dd HH:mm:ss.SSSZ' OR NUMBER or WORD. In your case it is matching NUMBER and picking out the first number in the message: 2021.

I suggest you try

grok {
    pattern_definitions => { "CUSTOMTIME" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}" } 
    match => { "message" =>"^%{CUSTOMTIME:tstmp}\|%{NUMBER:myevent}\|%{WORD:mylevel}*"
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.