How to calculate web page load times?

Elastic Stack Beats
1

Hi everyone,

I'm using Packetbeat to capture traffic from an Internet access network. How can I calculate and chart the times taken by individual users' devices to load web pages, over some time period? As you can see, since a web page contains many objects that are downloaded separately, and sometimes from different servers, the responsetime is given separately for each HTTP GET. In addition, the loadtime is blank.

Then, how can I calculate the page size, including all downloaded items for each page?

Many thanks!

web-page-download.png1366×738 67.8 KB

2

Have you had a look at the packetbeat HTTP dashboard? The dashboard for example create a histogram on response times. Using the dashboard/visualization you can use the filter bar to us search functionality to further filter the data you want to consider. With filters stored in the URL you can also bookmark a dashboard with filters applied.

3

Hi Steffens,

Yes - I have looked through the HTTP dashboard; but I cannot see a way to chart, ideally on a timeline:

  • the Web page load times (I don't mean the response times for individual HTTP GET operations)
  • size in MB of web pages (including all of their elements - HTML, CSS, GIF, etc)

Am I missing something?

Many thanks,
Ian

4

So you want to combine multiple related requests? This is not easily possible in a generic way with packetbeat, as packetbeat captures transactions only. You basically need a way to correlate multiple request/transactions into one page load. Maybe this can be done with some logstash community plugin?

5

As a passive observer of the traffic I wonder if it is possible to correlate all the requests to a single page load given that the same client could be loading multiple copies of the same page. I was thinking that you could possibly correlate a group of transactions based on the client ip, UserAgent, Referer and some time window.

6

Hi Andrew, Steffens,

I think Andrew is right - I couldn't think of any other way.

Could I perform this correlation using an Elasticsearch or Timelion query? If so, could you help me with the syntax for that please?

Otherwise are we talking of extending Packetbeat? Or would we need to write new beat?

Does Packetbeat's concept of a Flow help us?

Many thanks,
Ian

7

Flows might not really help. 1) The flow-id is not stored in HTTP transaction event yet. 2) browsers normally use N concurrent connections. 3) Some connections are re-used for additional requests.

With loading elements concurrently using multiple connections, request response times could not be easily added up.

Sounds like you want some integration with e.g. boomerang.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.