Filter logs using logstash

Elastic Stack Logstash
1

Hi!

I am using logstash to ingest some data from a sql DB. I am able to get the data in elastic but i want to get a substring from a filed and put it into another field. For example i have the following line:
{"Timestamp":"2018-10-17T10:36:26.1895556+03:00","Level":"Information","MessageTemplate":"MessageBusSubscriberService for message type {MessageType} is starting","Properties":{"MessageType":"ResetCache","SourceContext":"adjadsklaslk","CorrelationId":null}}
I want to get "Reset Cache" which is after "Message Type" and to put it in another field.

I tried with grok or mutate filter but i wasn't able to get that substring. Do you have any solution?

Thanks a lot!

2

Can you try this pattern and please find my result below
MessageType":"?(?[a-zA-Z]+)

Result
{
"MessageType": [
[
"ResetCache"
]
]
}

3

Hi!

Thanks for your reply! One more question: Which filter do i have to use for the pattern that you provide?

Thanks!

4

grok{
match => {"message" => "MessageType":"?(?[a-zA-Z]+)"}
}

5

I have the folloowing filter section:
filter {
grok {
match => {"logevent" => "MessageType":"?(?[a-zA-Z]+)"}
}
}
and when i run logstash i get the following error:
Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, } at line 16, column 40 (byte 522) after filter
I don'e see where is missing one of #, {, }.

Thanks!

6 
"MessageType":"?(?[a-zA-Z]+)"

If you are trying to match " in a field then either use ' or \" in the configuration

match => { "logevent" => '"MessageType":"?(?[a-zA-Z]+)"' }
7

Even if i use ' i still get an error:
exception=>"NoMethodError", :message=>"undefined method `unlock' for nil:NilClass"

Do you have any idea? Or is the pattern wrong?

Thanks!

8

The pattern is wrong because the markdown ate the field name

    match => {"message" => 'MessageType":"?(?<someField>[a-zA-Z]+)'}

However, if you are still getting "undefined method `unlock' for nil:NilClass" please provide the entire filter, becausei I would not expect to get that error from a grok.

Have you considered using a json filter?

json { source => "someColumn" }
9

Worked! Thanks a lot

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.