Filters in logstash for sending the logs to elastic search index

[Troy_Axthelm]

Based on the format of your Message field, it appears it is an incomplete json object. I notice that there is a closing '}' but not an opening one in the message field. If your message field was valid json, you would be able to parse that json and get values in a way similar to what you are trying.

Since this is not valid Json, you will have to use grok filters to get that value. If I am remembering correctly, you can use the predefined grok pattern for MAC addresses: %{MAC} and it should match the MAC address in the message. Example:

grok {
  match => { "message" => "%{MAC:macadd}" }
}

[Makra (Makara)]

Hi
Troy
Thanks for your reply, You have rightly pointed out about JSON's faulty structure but unfortunately many online JSON format tester will say it as correct. No way..have to used grok as advised by you.

Thanks again
Makara

[Troy_Axthelm]

From what you have posted, it looks like it taking the first part of your message and making it the value for the "PROGRAM" field. You would be better off if the "MESSAGE" field contained the full json object:

"MESSAGE": "{"EventTime":"2016-12-28 07:41:59","Hostname":"DESKTOP-7OTLF9V","Keywords":36028797018963968,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":1,"SourceName":"MYEVENTSOURCE","Task":0,"RecordNumber":2053,"ProcessID":0,"ThreadID":0,"Channel":"Application","Domain":"DESKTOP-7OTLF9V","AccountName":"devtst","UserID":"S-1-5-21-2657980916-529253927-1869581887-1001","AccountType":"User","Message":"My first log","Opcode":"Info","EventReceivedTime":"2016-12-28 07:42:00","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog","MAC":"00-0c-29-C3-E7-CE","Customer":"Contoso+test","Location":"DK-West"}"

This would allow you to parse that later on and get field values without having to use special grok patterns.