Remove part of message string

Elastic Stack Logstash
1

Hi there,

In the message field in kibana i am getting

2016-11-22 11:37:50,
943 app INFO: error_test - 82 test message
2016-11-22 11:37:50,
943 app ERROR: error_test - 83 test error log

As I have a separate @timestamp field, I think it would be redundant for my messages to contain the timestamp string. To make my logs neater, I want to remove the timestamp from my message.
I've tried using the drop filter, but I realized it's for fieldnames.
Any idea what filter i should use or how i can remove the timestamp from my message?

thanks!

2

I've tried using the drop filter, but I realized it's for fieldnames.

No, the drop filter drops whole events.

Any idea what filter i should use or how i can remove the timestamp from my message?

Use a mutate filter's gsub option or a grok filter.

3

Thank you for your reply,

I will use the mutate gsub filter, I'm having some difficulty coming up with the filter.

filter {
  mutate {
    gsub => ["message", "...", ""]
  }
}

What should i put for the timestamp 2016-11-22 11:37:50,943

Thanks!

4

This should work:

gsub => ["message", "\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} ", ""]
2 Likes
5

Works!! Thank you!

6

Hi, I'm able to remove the timestring, but Im facing a new problem.
Before using the above gsub filter, I was able to get the loglevels in my message(in my first post) into a field by using

mutate {
  split => { "message" => "," }
}

I'm able to get this

loglevel: INFO, ERROR

But after using the gsub filter, I'm only getting loglevel: INFO.
How can i solve this?

Thanks

7

Ignore my last post, I managed to fix it.

Thanks

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.