We had successfully set up the ELK stack to our production environment. We can also see the logs (logs are unstructured) output on our Kibana Server.
Everything is working fine for us. But the only thing we are concerned about is the messages in the kibana are structured for every single line written to a given log file.
Is there a way to merge( clubbing of lines ) the logs messages by max no of line ( in File beat before sending them to logstash or elastic search) which will be consider as 1 event in ElasticSearch / Kibana / Logstash.
Note: Just a note the log messages are unstructured (no particular regex pattern in them). So can't use (https://github.com/elastic/beats/issues/461). But I did tried the max-lines approach, yet the event in kibana shows a document for a single lines.
So is there a way to achieve this?