About Performance Measurement

The performance of a web site can be categorised by the two major parts:

• The web server performance. This is composed of the time to process the HTTP GET request and return the page contents. This time is affected by the speed of the processor, the size of the data, the number of concurrent requests etc. Dynamic content will usually take longer than static pages.
• The network. This time will be affected by the proximity to the monitoring point, the network capacity from the monitoring point to the web server and the amount of network traffic.

Netcraft's monitoring agents are placed at service providers in the USA, Canada, UK and several other European countries, so that the performance of the network can be determined from different points. At 15 minute intervals, each agent will make a HTTP GET request to each monitored site/url. The page returned by the HTTP GET request is not processed in anyway so the times do not include image downloads, style sheets and other embedded objects.

Successful Request

A request is successful when a monitor receives a complete HTTP response with a non-error HTTP status code within the timeout period (20 seconds).

Failed Request

A request is classified as failed when a single monitoring agent fails to receive a non-error HTTP response from the monitored site. A request may fail if the DNS resolution fails; or, if the TCP handshake fails to complete; if no bytes are received; or, if the HTTP response code represents an error condition.

Outage

Netcraft considers an outage to have occurred when there has been at least one failed request from every sampling point within each sampling interval.

Uptime

The opposite condition to an outage; the monitored site is responding to one or more monitoring agent with a non-error HTTP status code.

During the HTTP GET request the following stages are timed:

DNS

This is the time taken for the DNS lookup of the hostname. This will be affected by the local service provider name server as well as the authoritative name server for the hostname. It should be possible to determine the TTL (time to live) of the DNS entry, as at regular intervals the lookup will take much longer when a remote name server is queried. The time of the lookup is not included in the total time as it is very variable.

connect

This is the first phase of the HTTP GET request when the TCP/IP connection is setup to the remote server. This time will generally reflect the network latency time as the connection setup does not involve the web application. If the server is not responding or is under very heavy load this time will increase.

first byte

This is the time from when the last byte of the HTTP GET request is sent until the first byte of the response header is received. This is useful for determining how fast the web server is responding. This does depend on the application as some applications send the header before performing any processing while others complete all the processing before sending the response.

last byte

This is the time from when the last byte of the HTTP GET request is sent until the last byte of the page has been received. This value will depend on the performance of the web server, particularly if the page is dynamically generated, the size of the page and the network performance.

total

This is the time from when the HTTP GET request is started until the last byte of data is received. This includes the TCP connect time, the sending of the header and the receiving of the last byte, but not the DNS lookup time.

• DNS lookup failure — the DNS service was unavailable or failed to provide an address.
• Routing failure — a route to the target site could not be found.
• The request did not complete within 20 seconds

The data is presented as a series of graphs, one for each agent. The Y-axis of all the graphs is at the same scale and represents the time taken for the web server to respond. The X-axis is the time/date that the sample was made.

The X-axis scale shows 5 days of samples. Each pixel represents a 15 minute sample on the 5-day scale.

The Y-scale by default is calculated to show as much data as possible and is based on the average of the samples on the graph. All graphs have the same Y-scale.