Duration_in_millis shows too high values

[mathias]

Hi,

I am using ELK 5.6.3

I am looking at the delay of events and I can't understand the output values.

curl -s logstash:9600/_node/stats/pipeline | jq .pipeline.plugins.outputs
[
  {
    "id": "events_output_to_elasticsearch",
    "events": {
      "duration_in_millis": 29552,
      "in": 4512,
      "out": 4512
    },
    "name": "elasticsearch"
  },
  {
    "id": "beats_output_to_elasticsearch",
    "events": {
      "duration_in_millis": 18428,
      "in": 20251,
      "out": 20251
    },
    "name": "elasticsearch"
  }
]

The input delay is just 86 ms.

curl -s logstash:9600/_node/stats/pipeline | jq .pipeline.plugins.inputs[1]
{
  "id": "tcp_input_port_4029_eventdecoder",
  "events": {
    "out": 4979,
    "queue_push_duration_in_millis": 86
  },
  "name": "tcp"
}

For the events_output is is 29 seconds.
When I hit the button to generate an event it takes approximately 2 seconds before I see the event in Kibana.
Kibana is refreshing once per second.

Is there an explanation to the high number duration_in_millis?

Have I misunderstood what duration_in_millis means or is this a bug?

Kinds Regards
Mathias

[Christian_Dahlqvist]

I believe duration_in_millis is the total time across the number of events listed. 29552 ms across 4512 events is 6.54 ms per event. As multiple events are sent per batch and multiple batches can be sent in parallel, this does not directly relate to elapsed wall-time.

[mathias]

Thanks,

That explains a lot.

Br Mathias