Packetbeat recording high responce time for connections

[pratikshatiwari]

Hello

Environment detail -
Packetbeat 8.3 installed on Elasticsearch 8.3 server to monitor the completed connections duration. where we want to know in what time duration the incoming query search connection got completed

To monitor the same we have enabled the network.flow in packetbeat.yml

Problem statement -
We are facing issue where packetbeat is document more time in event.duration then actual time.

We have search API (built on different machine) setup to perform search query against the elasticsearch cluster, so whenever we perform the search query from search API it record the time of 1.5s but the same query record the time in elasticsearch cluster more than 3s which should be smaller & under few ms expected

Can we please get any support here on why elastocsearch packetbeat is recording high responce time as the event duration should count only the time taken from elasticsearch to respond connection which ideally below 300ms

Regards
Pratiksha

[pratikshatiwari]

hello Team,

Any suggestion will be helpful

Regards
Pratiksha

[mtojek (Marcin Tojek)]

Is packetbeat in the same network as Elasticsearch? Maybe it's a normal network latency in your setup.

[pratikshatiwari]

Hello @mtojek

Thanks for reply

Yes packetbeat is on same network infact it is installed on elasticsearch server we need to monitor the time duration taken by elasticsearch server to respond to a query

E.g when we perform the search execution from dev tools it is reporting approx 200ms or less

But when the similar request we are executing from elasticsearch search api client packetbeat recording the responce time more than 6s

This behaviour we are observing since we upgrade our setup from v7.14 to 8.3

Prior upgrade similar query used to record in packetbeat data as 200ms or less even

Regard
Pratiksha