Specifies the date format

Elastic Stack Logstash
1

A timestamp that specifies the format

07/20/20 00:00:08
match => ["timestamp","mm/dd/yy HH:mm:ss"]

after test result grokparsefailure

how to slove?

2

Unfortunately I don't understand your question.
Either you tried to use grok with a date pattern, which would be wrong as you need a date filter for that, or what you posted is your date filter and that won't help us at all to solve your problem because your error occurs at a grok filter.
Please post a more detailed response with your full data and configuration.

3

1st quesion:
source filename: /home/elk/hzam_1_perf.log,i want only get hzam,how to slove? Is there any other way??

match-mutate
match-mutate901×485 6.95 KB

2nd question:
log content:
07/20/20 00:00:08 INFO infocity:sh(city/CityInfoBusiness.cpp:134)

The final field that i want to enter into ES

07/20/20 00:00:08 INFO infocity:sh hzam(1st can get)

07/20/20 00:00:08 the date format how to filter match

4

Please don't post pictures of text. That's so inconvenient :frowning: Copy your configuration and insert it here as a code block (</> button).

  1. If you use Ruby you can extract it with event.set("reg_str", event.get("path").split("/")[-1].split("_")[0].split(".")[0]). If you use grok, the regex is ^(.*\/|^)(?<reg_str>.*?)(_|\.|$) (Both solutions should work even if the path contains no /,_ or . at all)
  2. The pattern for your date filter is MM/dd/yy HH:mm:ss
5

good job,thanks for yours

log content:

07/20/20 09:28:57 INFO deletecity:bj(city/CityDeleteBusiness.cpp:55)

source: /home/ops/log/.../abc_4_perf.log20200802

The end result you want

reg_str: abc
timestamp:
loglevel: INFO
action: deletecity
city: bj

then how do i include timestamp format
and how to removefield infoother

this is my config file

       filter {
        grok {  match => {   "message" => "%{DATE_US:elk_day} %{TIME:elk_time} %{LOGLEVEL:loglevel} %{GREEDYDATA:action}\:%{GREEDYDATA:city}\(%{NOTSPACE:infoother}\)"
            }
        }
 
    date {
         match => ["elk_day","MM/dd/yy Z"]
         match => ["elk_time","HH:mm:ss Z"]
         locale => "en"
         timezone => "Asia/Shanghai"
    }
    ruby {
        code => 'event.set("reg_str",event.get("source").split("/")[-1].split("-")[0].split("_")[0])'
    }
    mutate {
        remove_field => [ "{infoother}" ]
    }
}
6
  1. You can't just define the match option of the date filter multiple times (you should actually never have the same option multiple times in one filter). You need one string to be parsed, which means a) building a combined string of date and time before the date filter or b) not separating them in the first place, but getting them both together by using DATESTAMP in the grok filter.
    https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns
  2. Why are there curly brackets in your remove_field? Because of these it doesn't work. The field name is infoother, not {infoother}. And if you don't want to have that field you could just leave it out in your grok pattern and end the pattern after the (. Then there wouldn't even be a field to delete.
7

I would say "should not" rather than "can't". Unfortunately logstash is very forgiving of this. It will merge them. It will .to_h or .to_a, or even .to_s whenever it needs to do so to force the initial parse to conform to the required syntax.

In my experience it often does not merge them in the way that any reasonable person would expect.

1 Like
8

Oh thanks. Interesting to know O.o

9

I'm sorry, I don't know how to match the date format,could you help me ?

10

At which point do you struggle? You could grab it as %{DATESTAMP:elk_datetime} and then parse it with MM/dd/yy HH:mm:ss like I had suggested earlier.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.