Multiple logstash filters with grok and mutate

Elastic Stack Logstash
1

Hi everybody,

I working on a logstash filter that contains both grok and mutate. The idea is to create two fields field1 and field2 with grok and remove some information from a field message
Here is my configuration

grok {
  match => [
    "message",
    "^*(?<field1>[^*]+)%{UUID}"
  ]
}
grok{
  match => [
   "message",
   "^*%{UUID}(?<field2>[^*]+)applogs"
  ]
}
mutate {
  gsub => [
    # replace applogs with nothing
    "message", "%{field2}applogs", ""
  ]
}

This configure works but I have this issue that when the following event is coming in

berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14applogs 2017-02-10 05:34:24 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14, user); groups with view permissions: Set(); users  with modify permissions: Set(berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14, user); groups with modify permissions: Set()

The field1 is create as such

berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14applogs·2017-02-10·05:34:24·INFO··SecurityManager:54·-·SecurityManager:·authentication·disabled;·ui·acls·disabled;·users··with·view·permissions:·Set(berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14,·user);·groups·with·view·permissions:·Set();·users··with·modify·permissions:·Set(berry

Meanwhile when an exent

berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14applogs 2017-02-10 05:34:24 INFO  Utils:54 - Successfully started service 'Driver' on port 41958.

The field1 is create with

berry

Which is what I expect for as value for field1

My guess is that the issue is because UUID is presence twice in the message. I have tried many combinations but I could not get it work.

Another thing that I will like to do is remove the berryf050d08a-9e3e-4926-a5b0-8624236a09aaberry-app14applogs once the two grok have successfully done their work.

I have tried this

 mutate {
  gsub => [
    # replace %{field1}%{UUID}%{field2}applogs with nothing
    "message", "%{field1}*%{field2}applogs", ""
  ]
}

But this is not working for me.

Any clue please?
Thanks.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.