I am thinking of implementing Packetbeats to send telecom grade packet traces to eleasticsearch. In a telecom network, we are dealing with different types of protocol stacks such as SS7 (including GSM MAP), and TCP including SIP, Diameter. May I know .... packetbeats able to forward those type of packets to elasticsearch? or is there a limitation?
Please share with me if you have a guide to go through?
Packetbeat does not support the decoding of very many protocols. You will be better off using TShark (the CLI version of wireshark) as it has support for the largest number of protocols. It has an Elasticsearch output mode, which sends JSON to Elasticsearch. There are a few things to consider...
Obviously the volume of data can be considerable, so you will want to do some pruning/cleanup of the data provided to keep only that which you really need. Logstash will be too slow in this case. You will want to use Elasticsearch ingest pipelines instead.
Consider writing each protocol to its own index. This will help to keep index sizes managable, and reduce the total number of fields per index. The latter is necessary because the number of fields can explode into the thousands and cause problems working with the data in Kibana.
We have a lot of experience working with the Elastic Stack for such network analytics use-cases. If you find yourself looking for outside help, send me a PM.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.