With the pricing models used in cloud for ingress and egress it has become relevant to understand how much traffic will occur over the course of a year before actually implementing elastic.
With this in mind, if I have an application that generates 8 GB of standard application logs (timestamp, type of error and one-liner description of the event). How much bandwidth will Filebeat need to transfer those 8 GB of log text? (considering compression, etc.)
You can run Filebeat on your logs and enable the HTTP endpoint in filebeat.yml using http.enabled: true.
This will then let you look at Filebeat's internal metrics by visiting http://localhost:5066/stats. In the response JSON, if you look under the libbeat.output.write key, you'll see how many bytes are being written over the wire by Filebeat.
This is helpful, but this is when things are running. I meant more as a benchmark or best practice. Here we currently are at a design and MVP phase where we need to be able to estimate the above to determine the feasibility of implementing the stack (to define the non-functional requirement).
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.