# Http responsetime unrealistic results
**URL:** https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276
**Category:** Beats
**Tags:** packetbeat
**Created:** [January 27, 2016, 6:35pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276 "2016-01-27T18:35:10Z")
**Posts on this page:** 18
**Page:** 1
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [January 27, 2016, 6:35pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/1 "2016-01-27T18:35:10Z")
Hi,
I am recording data from a number of Elasticsearch cluster. For some queries, I get huge unrealistic response time numbers (3 mins).
When I run the queries on the clusters, they return in 100ms~.
note: I didn't use PacketBeat template file.
can you please explain why this can happen? how does packetbeat calculate the response time (is it the "took" field from Elasticsearch response?).
---
### Author:  [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [January 27, 2016, 7:31pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/2 "2016-01-27T19:31:11Z")
Packetbeat doesn't use the "took" field, instead it looks at the timestamp of the request and the timestamp of the response. At the moment it actually doesn't look into the payload at all, so it doesn't know that you have Elasticsearch running, just that there's an application using HTTP.
If you have the feeling that the values are not realistic, it could be that the request is matched with the response from another request. We call this a correlation problem. Causes could be packet drops or parsing errors.
One way to check for correlation issues is to configure packetbeat to store both the full request and the response (`send_request: true`, `send_response: false` and `include_body_for: ["application/json"]`), then for the transactions that have unrealistic times, check if the two seem to match.
---
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [January 28, 2016, 11:33am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/3 "2016-01-28T11:33:43Z")
did you mean send\_request: true, send\_response: true?
because I have this configured already. the "include body for" too.
I do see now that the request and the response don't match (I get a different response when running the query on ElasticSearch directly).
How can I solve this correlation problem?
If this problem has no instant solution, can I use Logstash or some other tool to parse the response from ElasticSearch, and the "took" field and other fields into the JSON object?
---
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [February 4, 2016, 11:31am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/4 "2016-02-04T11:31:56Z")
Tudor, can you please help me fix this bug?
i can't use this tool because of the above.
---
### Author:  [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [February 4, 2016, 11:53am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/5 "2016-02-04T11:53:54Z")
Next step would be to identify why is the miss-correlation happening. The most likely reason is that Packetbeat drops messages when reading form the network interface.
Some questions:
- How often does the miss-correlation happen? Maybe try putting them on a graph in Kibana (all transaction with responsetime \> 30s) and see if they are clustered in some periods or randomly distributed.
- What sniffer type do you use in Packetbeat? The defualt pcap or af\_packet? Would be good to post the full config.
- How many requests per second is Packetbeat seeing
- If you do `ifconfig eth0` where eth0 is the interface where it is sniffing, do you see any drops or errors?
---
### Author:  [@tudor](https://discuss.elastic.co/u/tudor)
#### Post date: [February 4, 2016, 11:55am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/6 "2016-02-04T11:55:40Z")
> [@jonatanzafar59](#):
>
> If this problem has no instant solution, can I use Logstash or some other tool to parse the response from Elasticsearch, and the "took" field and other fields into the JSON object?
You could try it like that, but probably won't be very easy.
Depending on what you are trying to accomplish, the [slow query log](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html) and Marvel might also be helpful.
---
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [February 5, 2016, 10:32pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/7 "2016-02-05T22:32:27Z")
Hi Tudor, Thank you for the answer.
1. this occurs in a randon time frame (I checked the the transactions with a negative response time, which i guess comes from the root cause)
2. the config:
interfaces:
device: any
protocols:
http:
ports: [9200]
send\_request: true
send\_response: true
include\_body\_for: ["text/html", "application/json"]
mysql:
ports: [3306]
mongodb:
ports: [27017, 27019]
send\_request: true
send\_response: true
output:
redis:
enabled: true
host: "redishostname"
port: 6379
1. i don't know for sure, because this is still in test environment and i delete yersterday's indices.
2. I do see drops on some servers, BUT, when i run responsetime \< 0, and visualize in kibana for the result's split across the servers, i get that most of the transactions came from servers without any drops.
This drops makes me worry, does the packetbeat has to do anything with the drops? those servers have the same hardware as production, and with the same exact traffic, i get drops on the test and not on prod.
---
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [February 10, 2016, 12:18am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/8 "2016-02-10T00:18:55Z")
Tudor, what should we do about it?
I used Logstash to parse the response, and use the took data as a new field, but I have a lot of transactions where the request doesn't match the response.
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 10, 2016, 3:44pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/9 "2016-02-10T15:44:20Z")
Is there a chance to get a trace with raw network packets? I'd like to have a look and see if we can improve correlation.
---
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [February 11, 2016, 11:29am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/10 "2016-02-11T11:29:45Z")
I can't get the production traffic out.
Ask me anything, I really need the traffic to correlate correctly.
---
### Author:  [@jonatanzafar59](https://discuss.elastic.co/u/jonatanzafar59)
#### Post date: [February 15, 2016, 5:43pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/11 "2016-02-15T17:43:15Z")
Issue seem (!) to be resolved after upgrading (packetbeat's) Elasticsearch to 2.2.0 version (no sure why this is related) .
Edit: \*\* this did not solve the problem, just mitigated it
---
### Author:  [@Rachit\_Puri](https://discuss.elastic.co/u/Rachit_Puri)
#### Post date: [April 22, 2016, 12:42pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/12 "2016-04-22T12:42:24Z")
Any updates on this problem, i can see response time to be 0,1 for most of my traffic
---
### Author:  [@harshafrnd4u](https://discuss.elastic.co/u/harshafrnd4u)
#### Post date: [June 28, 2016, 11:22am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/13 "2016-06-28T11:22:24Z")
Even I see that http responsetimes in either 0,1 and why not decimals? Is it rounding off?
---
### Author:  [@Akashi\_Seih](https://discuss.elastic.co/u/Akashi_Seih)
#### Post date: [January 4, 2017, 1:12pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/14 "2017-01-04T13:12:25Z")
> [@harshafrnd4u](#):
>
> Even I see that http responsetimes in either 0,1 and why not decimals? Is it rounding off?
I have the same problem. The response time for queries is mostly 0. Is this a bug?
---
### Author:  [@harshafrnd4u](https://discuss.elastic.co/u/harshafrnd4u)
#### Post date: [January 4, 2017, 1:42pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/15 "2017-01-04T13:42:44Z")
Its been long I tested, I couldn't solve that at that time. I didn't dig deep into it, not sure if it is a bug.
---
### Author:  [@harshafrnd4u](https://discuss.elastic.co/u/harshafrnd4u)
#### Post date: [March 17, 2017, 11:28am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/16 "2017-03-17T11:28:54Z")
Hi, Are you able to figure out solution to this problem?
---
### Author:  [@harshafrnd4u](https://discuss.elastic.co/u/harshafrnd4u)
#### Post date: [March 17, 2017, 11:33am UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/17 "2017-03-17T11:33:46Z")
@Akashi_Seih @jonatanzafar59 @Rachit_Puri Hi, Can you please help me if anyone if figured out the solution to this problem?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 9:49pm UTC](https://discuss.elastic.co/t/http-responsetime-unrealistic-results/40276/18 "2017-07-05T21:49:48Z")