Parsing different log format from same application (input)

I started reading about Logstash some days ago to find out wether it is suitable for my usecase. There is one critical requirement for which I haven't found a solution yet.

My application already writes logs to files using log4j2. All these log messages have the same structure (e.g. LEVEL TIME MESSAGE) and are written human readable (e.g. User XY has deleted item Z). All the log events are generated from different actions in the application, so they are most likely not related to each other and represent different information.

Is Logstash able to parse each message individually? Let's say I have a message like "XY has been deleted by Z" and a message like "User ABC signed up" or "Password of user DEF was reset". Each of these messages have some core information which needs to be extracted into fields. In case of my last example I want something like action = password_delete, user = DEF. It is important to note that not every log message has an "action" or a "user".

I hope it's clear what I'm asking. If not I'll try to explain in more detail.

1 Like

You can use a grok filter with multiple expressions that attempt to match the different kinds of human-readable message. Those expressions will be tried in order until one matches (or none of them does).

1 Like