Parsing with logstash


(Samir) #1

Hello EveryOne ,

I need your help to extract information from a document contained in a log.
The document is as follows :

2017-05-10T14:28:42.387 Int 22000 ##### EI_COFBE_SICRC04_GestionHO_SSTR-v8 - 01df028ded513257
_I_I_01df028ded513257 [09:04] ASSIGN: zFlow(LOCAL) <- STRING: "379000:376000:COFBE_070_FR_CIBLE:HO:COFBE_DEVELOPPEMENT_FR:19:TO:COFBE_OUTSOURCER_FR:8:TO:COFBE_SEDUCTION_FR:6:5307:OK:"
_I_I_01df028ded513257 [09:04] ASSIGN: iTimeStamp(LOCAL) <- INTEGER: 413961

I want to extrat only what is highlighted.
I try this filter :

match => {"message" => "zFlow(LOCAL) <- STRING: %{GREEDYDATA:Flow}[\n]"}

He sends me back what I want but also the rest of the document.
How could I remove the rest of the document ?

Thanks for your help !


(Magnus Bäck) #2

You can e.g. use

zFlow\(LOCAL\) <- STRING: "(?<Flow>[^"]*)"

to extract everything inbetween the double quotes. Note that you'll have to make the grok expression string a single-quoted string (it's currently double-quoted). Another option is to use the QS grok pattern but it keeps the surrounding double quotes in the extracted string.


(Samir) #3

Thank you for your help.

I changed my filter by the one you provided :
match => {"message" => "zFlow\(LOCAL\) <- STRING: "(?<Flow>[^"]*)"}

But it does not work. The logstash configuration file no longer launches.


(Magnus Bäck) #4

As I said: Note that you'll have to make the grok expression string a single-quoted string (it's currently double-quoted). That means this:

match => {"message" => 'zFlow\(LOCAL\) <- STRING: "(?<Flow>[^"]*)'}

But it does not work. The logstash configuration file no longer launches.

In this particular case I was able to spot the error anyway, but please try to anticipate the questions are you going to get. If Logstash doesn't start we're going to want to see the logs.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.