I need to design a system to read different types of Payloads and a Json Log file.
I am able to read both of them and create Indexes in ElS (Filebeat--> Logstash-->ES-->Kiabana) individually.
But now I have two problems
- These Payloads are in different formats some are in JSON, some are in XML, they dont have file extensions or anything, we can identiy them only by seeing the content in it. can any filter help me to Identify the content ? I want to identify the content type so that Kiabana can give me better viewing.
- I have another filebeat which reads Json log file, this log file has
payloadfor that transaction in local disk, which I am reading in different filebeat or ES index, can I merge these two filebeats and create single index where I can see the payload along with the Transaction log.
Are these two challenges can be solved ? if so please direct me to some documentation.