Hi guys, I am new to Elasticsearch... I am using filebeat to ship the multiple line unstructured messages to logstash and use grok filter to structure the data. However, I noticed in the application logs, between messages, some messages have less field/value pair, some messages have more field/value pair, the grok
patterns doesn't work for all messages... so of the messages can't be structured. Could someone help me and give me a right direction for the solution?
Example:
message 1:
04:45:01.633 - R (1,2) msglen:89 New
Timestamp:109660 ItemCode:TEST1 ListingDate:20100102 Expiry:0 Term:
message 2:
04:45:01.633 - R (5,9) msglen:97 DELETE
Timestamp:20988 ItemCode:TEST2 ListingType:CIN ListingDate:20160102 Term: Processingdate: 2017-0108
Many thanks!