Creating a sub field of GROK filter pattern

Hi Everyone,

i have a logstash configuration to parse my current logs but i would like to create a sub field from my message field and that data should be parsed as a fieldname, please suggest how that is possible:

For example in the following logs how can i make a separate field for PosErr_Advancing

Oct 10 09:05:29 ff303-srv1 MC: ERROR {33417} [RDSA]        Lim='PosErr_Advancing' IsPosition=5.964 SetPosition=nan IsSpeed=-44098.585 IsAcceleration=-11024646.148 Load=-1.647 TriggerDelay=0.000s SpeedLim=0.000% Ramp=5.000m/s2 SafeState=JobDeactivated ResetDelay=1.000s BrakeInstantly=N AffectSa(truncated)

my current log pattern looks like this :slight_smile:

grok {
        match => {"message" => '(?<Timestamp>[\w\s\d\:]+)\s(?<Server name>[\w\d]+)\-(?<Server number>[\w\d]+)\s(?<Product>[\w\:]+)\s(?<Severity>[\w\s\{\d\}]+)\s(?<body>[\w\s\w\=\"\d\:\s\"\?\'\/\.\-\,\{\}\[\]\(\)]+)'

Personally I would not grok that, I would dissect and then use kv.

Hi @Badger thanks and i used the dissect and kv for this but at some places in log files because of a space timestamp is read into the next field, i will add my config and log files below, please check and suggest how i can read spaces if it appears sometimes:


 filter {
        dissect { mapping => { "message" => "%{timestamp1} %{+timestamp1} %{+timestamp1} %{Theatre} %{Product} %{Severity} %{Body}" 

"Body" => "%{1} %{2} %{3} %{4} %{5} %{6} %{7}" } }

        kv { source => "Body" }
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

and the log data is like below :

Jul  8 05:01:07 ff302-srv1 MC:  debug  {0} [Equip]   Adding equipment="gio" Id=1
Jul  8 05:01:07 ff302-srv1 MC:  debug  {0} [Equip]   Adding equipment="rev" Id=2

sometime there is log data where i have some space within timestamp and at that time date is being read in the theatre field instead of timestamp, please check and share your views on this

If the problem is the extra space between Jul and 8 then change the mapping from

"%{timestamp1} %{+timestamp1} %{+timestamp1} %{Theatre}...


"%{timestamp1->} %{+timestamp1} %{+timestamp1} %{Theatre}...
1 Like

thanks @Badger it worked :grinning:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.