How to find the time taken to upload/write a file from filebeat to logstash to elastic search(pipeline)?

So I have a log file that is taken as an input by filebeat which is then outputted to logstash and from there to Elasticsearch. I need to calculate the time required/elapsed from the time I start filbeat and its reading process till the time it reaches Elasticsearch.

The pipeline is logfile-->filebeat-->logstash-->els. I need to find time elapsed from filbeat to els. I'm new to ELK and don't know how to use grok or similar features yet.

I have configured filebeat.yml as follows:


  • type: log paths:
  • /home/user/us/logs/lgfile.json output.logstash: hosts: ["localhost:port"]

and logstash conf file as:

input {

    beats {

        port => "5044"



output {

    stdout { codec => rubydebug }
    elasticsearch {
    hosts => ["localhost:port"]
    index => "elkpipego"


So far I have manually done this time calculation using a stopwatch from the time logstash reads the first msg and then used the difference with the timestamp of the last message. But this is clearly not so accurate and I would like to know if there's a way using any feature/grok/tool like jmeter/esrally or anything where I could accurately find the exact time elapsed till the last message is stashed on Elasticsearch? TIA

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.