Hi all.
I would like to test whether an EBK (es, beats, kibana) stack could be setup to work in my situation. I am very new to Beats and only was introduced to it in the past workshop I recently attended.
We have a custom binary log that our app generates, which in turn requires a custom binary decoder to extract data from.
Is there anyway I can use File Beats or some sort of other Beats module that can execute the decoder and send output to ES?
Example syntax of decoder:
./binary_decoder -env dev -binary_log '/opt/data/app/binary_log_010203020302_binary.log'
Redirecting output to file populates something like:
dev.myApp.coreSlave1113.component.backend.rpc.GetSomeParams.count 458 1549078080
dev.myApp.coreSlave1113.component.backend.rpc.GetSomeParams.successcount 458 1549078080
dev.myApp.coreSlave1113.component.backend.rpc.GetSomeParams.failcount 0 1549078080
dev.myApp.coreSlave1113.component.backend.rpc.GetSomeParams.authfailcount 0 1549078080
dev.myApp.coreSlave1113.component.backend.rpc.GetSomeParams.totaltime 545294 1549078080
then formulate desired data structure via regex:
*.*.*.component.*.rpc.*.count
name="myApp_rpc_count"
myApp_environment="${1}"
myApp_service="${3}"
myApp_component="${7}"
myApp_rpc="${8}"
(The above two blocks is from the graphite_exporter that we currently have setup)
I would like to send the above to ES directly via Beats, rather than hack something additional together where I need to create some wrapper to extract files to some dir where file beats can read of off, and have to worry about implementing some mechanism where I then need clean the files outputted by this hack so that my partition doesn't run out of space etc.
I just found some pages on Logstash and a graphite plugin. I would also like to avoid having to setup Logstash as that would introduce yet another tech I need to setup and install.
Thank you in advance...