• Performance measurement
• Registration and login
• Profiles

The performance of a web site can be categorised by the two major parts

• The web server performance. This is composed of the time to process the http GET request and return the page contents. This time is affected by the speed of the processor, the size of the data, the number of concurrent requests etc. Dynamic content will usually take longer than static pages.
• The network. This time will be affected by the proximity to the monitoring point, the network capacity from the monitoring point to the web server and the amount of network traffic.

The monitoring agents are placed at different service providers in the UK,USA and Europe so that the performance of the network can be determined from different points.

At 15 minute intervals, each agent will make a http GET request to each monitored site/url. The page returned by the http GET request is not processed in anyway so the times do not include image downloads, style sheets and other embedded objects.

During the http GET request the following stages are timed

dns

Time for the DNS lookup of the hostname. This will be affected by the local service providers name server as well as the authorative name server for the hostname. It should be possible to determine the TTL (time to live) of the DNS entry as at regular intervals the lookup will take much longer when a remote name server is queried. The time of the lookup is not included in the total time as it is very variable.

connect

This is the first phase of the http GET request when the TCP/IP connection is setup to the remote server. This time will generally reflect the network latency time as the connection setup does not involve the web application. If the server is not responding or is under very heavy load this time will increase.

first byte

This is the time from when the last byte of the http GET request is sent until the first byte of of the response header is received. This is useful for determining how fast the webserver is responding. This does depend on the application as some applications send the header before performing the any processing, others complete all the processing before sending the response.

last byte

This is the time from when the last byte of the http GET request is sent until the last byte of the page has been received. This value will depend on the performance of the web server, particularly if the page is dynamically generated, the size of the page and the network performance.

total

This is the time from when the http GET request is started until the last byte of data is received. This includes the TCP connect time, the sending of the header and the receiving of the last byte, but not the dns lookup time.

The data is presented as a series of graphs, one for each agent. The Y-axis of all the graphs is at the same scale and represents the time taken for the webserver to respond. The X-axis is the time/date that the sample was made.

The X-axis scale can show 5 or 20 days of samples. Each pixel represents a 15 minute sample on the 5-day scale and 1 hour samples on the 20-day scale. The value of the 1-hour samples are the average of the samples within that hour. The 15-minute samples are summarised into hourly values after a few days.

The Y-scale by default is calculated to show as much data as possible and is based on the average of the samples on the graph. All graphs have the same Y-scale.

Registration and login

The performance data is only available to users that have contacted Netcraft and arranged access. The registration process only requires an email address (used as your username) and a password.

In order to verify that your email address is valid we will send an email that contains a special url that when visited will confirm that you can read email at the email address that is your username.

As soon as you have registered you may login and access any pages that you are entitled to. If after 2 days the special url has not been visited then the username will be deleted.

Profiles

Users can create a list of sites or urls that they wish to monitor. The results from this list can be viewed as a table so that sites can be compared. The table can be sorted by any of the columns i.e. site name, connect time, outage time, percentage of errors etc.

Users may also enable email alerts when ever a site on a profile is not accessible from all collectors during a 30-minute interval.

Sites may be added and removed from the profile by using the Edit profile link in the navigation bar. This page also allows the enabling and disabling of email alerts.

The navigation bar will contain links to all the profiles to which you are allowed access.

The edit and view profile pages allow the sites displayed to be limited to smaller set by entering a pattern into the find site box on the pages. This is a simple textual pattern that will be used to remove sites that don't have the pattern in.