Creating a sub field of GROK filter pattern

Elastic Stack Logstash
1

Hi Everyone,

i have a logstash configuration to parse my current logs but i would like to create a sub field from my message field and that data should be parsed as a fieldname, please suggest how that is possible:

For example in the following logs how can i make a separate field for PosErr_Advancing

Oct 10 09:05:29 ff303-srv1 MC: ERROR {33417} [RDSA]        Lim='PosErr_Advancing' IsPosition=5.964 SetPosition=nan IsSpeed=-44098.585 IsAcceleration=-11024646.148 Load=-1.647 TriggerDelay=0.000s SpeedLim=0.000% Ramp=5.000m/s2 SafeState=JobDeactivated ResetDelay=1.000s BrakeInstantly=N AffectSa(truncated)

my current log pattern looks like this :slight_smile:

grok {
        match => {"message" => '(?<Timestamp>[\w\s\d\:]+)\s(?<Server name>[\w\d]+)\-(?<Server number>[\w\d]+)\s(?<Product>[\w\:]+)\s(?<Severity>[\w\s\{\d\}]+)\s(?<body>[\w\s\w\=\"\d\:\s\"\?\'\/\.\-\,\{\}\[\]\(\)]+)'
        }
    }}
2

Personally I would not grok that, I would dissect and then use kv.

3

Hi @Badger thanks and i used the dissect and kv for this but at some places in log files because of a space timestamp is read into the next field, i will add my config and log files below, please check and suggest how i can read spaces if it appears sometimes:

Config


 filter {
    
        dissect { mapping => { "message" => "%{timestamp1} %{+timestamp1} %{+timestamp1} %{Theatre} %{Product} %{Severity} %{Body}" 

"Body" => "%{1} %{2} %{3} %{4} %{5} %{6} %{7}" } }
       

        kv { source => "Body" }
    }
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

and the log data is like below :

Jul  8 05:01:07 ff302-srv1 MC:  debug  {0} [Equip]   Adding equipment="gio" Id=1
Jul  8 05:01:07 ff302-srv1 MC:  debug  {0} [Equip]   Adding equipment="rev" Id=2

sometime there is log data where i have some space within timestamp and at that time date is being read in the theatre field instead of timestamp, please check and share your views on this

4

If the problem is the extra space between Jul and 8 then change the mapping from

"%{timestamp1} %{+timestamp1} %{+timestamp1} %{Theatre}...

to

"%{timestamp1->} %{+timestamp1} %{+timestamp1} %{Theatre}...
1 Like
5

thanks @Badger it worked :grinning:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.