The performance of a web site can be categorised by the two major parts
The web server performance. This is composed of the time to
process the http GET request and return the page contents. This time is
affected by the speed of the processor, the size of the data, the number
of concurrent requests etc. Dynamic content will usually take longer
than static pages.
The network. This time will be affected by the proximity to the
monitoring point, the network capacity from the monitoring point to the
web server and the amount of network traffic.
Netcraft's monitoring
agents are placed at service providers in the USA and the UK and the so
that the performance of the network can be determined from different
points. At 15 minute intervals, each agent will make a http GET request
to each monitored site/url. The page returned by the http GET request is
not processed in anyway so the times do not include image downloads,
style sheets and other embedded objects.
During the http GET request the following stages are timed
dns
This is the Time taken for the DNS lookup of the hostname. This will be
affected by the local service provider name server as well as the
authorative name server for the hostname. It should be possible to
determine the TTL (time to live) of the DNS entry, as at regular
intervals the lookup will take much longer when a remote name server is
queried. The time of the lookup is not included in the total time as it
is very variable.
connect
This is the first phase of the http GET request when the TCP/IP
connection is setup to the remote server. This time will generally
reflect the network latency time as the connection setup does not
involve the web application. If the server is not responding or is under
very heavy load this time will increase.
first byte
This is the time from when the last byte of the http GET request is sent
until the first byte of the response header is received. This is useful
for determining how fast the web server is responding. This does depend
on the application as some applications send the header before
performing any processing while others complete all the processing
before sending the response.
last byte
This is the time from when the last byte of the http GET request is sent
until the last byte of the page has been received. This value will
depend on the performance of the web server, particularly if the page is
dynamically generated, the size of the page and the network performance.
total
This is the time from when the http GET request is started until the
last byte of data is received. This includes the TCP connect time, the
sending of the header and the receiving of the last byte, but not the
dns lookup time.
Typical reasons for not being able to get a sample are:
DNS lookup failure - the DNS service was unavailable or failed
to provide an address.
Routing failure - a route to the target site could not be found.
The request did not complete within 20 seconds
Graphs of Individual Sites
The data is presented as a series of graphs, one for each agent.
The Y-axis of all the graphs is at the same scale and represents the
time taken for the web server to respond. The X-axis is the time/date
that the sample was made.
The X-axis scale shows 5 days of samples. Each pixel represents
a 15 minute sample on the 5-day scale.
The Y-scale by default is calculated to show as much data as possible
and is based on the average of the samples on the graph. All graphs have
the same Y-scale.
