This article looks at one way that we have made Programmer's Heaven load faster, especially for our users outside of the US, using a service called CacheFly.

Understanding the problem

Web requests are made using the HTTP protocol. A web browser sends a HTTP request - a chunk of text specifying what file is required - over the network to the web server. The web server then gets this file and sends it over the network to the web browser.

There are two network related factors that affect the time that the request takes. One is latency: how long does it take for a bit of data to get from the browser to the server or vice versa. The other is the transfer rate: how many bits can we send or receive per unit time.

HTTP is layered on top of TCP/IP, the two protocols that together provide routing, error control and flow control. When you want to make a TCP/IP connection to a remote computer, you sent a packet to it. The remote computer responds with another packet stating that the connection is now open. This process is required to agree on sequence numbers. After that, packets of data can be sent - the data being the HTTP request. Finally, the web server sends something back.

The thing to note here is that before we even get back any useful data we have had to wait at least four times the latency of the network between the user and the web server. Therefore, decreasing this latency should give us notable benefits.

Introducing CacheFly

The Programmer's Heaven servers are located in a datacenter in the west of the US. We serve all our content from one location. CacheFly, by contrast, take your data and put it on multiple servers located at different places throughout the world. When a user makes a request for a file that CacheFly are hosting, they are directed to the server that is closest to them and should, in theory, have the lowest latency. I did some experiments to confirm this.

Checking out the ping times

The ping tool measures the round trip time - that is, the time for a packet of data to go from my computer to another and then be bounced back again. This is approximately twice the latency of the network (though it's never that simple; routing configuration means that packets often take different routes over the internet in each direction). I'm based in the UK, so for me the ping times to the Programmer's Heaven server are rather long:

C:\Documents and Settings\Jonathan>ping programmersheaven.com
Pinging programmersheaven.com [68.142.114.70] with 32 bytes of data:
Reply from 68.142.114.70: bytes=32 time=169ms TTL=111
Reply from 68.142.114.70: bytes=32 time=173ms TTL=111
Reply from 68.142.114.70: bytes=32 time=175ms TTL=111
Reply from 68.142.114.70: bytes=32 time=170ms TTL=111
Ping statistics for 68.142.114.70:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 169ms, Maximum = 175ms, Average = 171ms

Knowing CacheFly claim to have a server in the UK, I reasoned that I would get notably faster ping times. This was confirmed - the difference was a factor of seven!

C:\Documents and Settings\Jonathan>ping pheaven.cachefly.net
Pinging pheaven.cachefly.net [205.234.175.175] with 32 bytes of data:
Reply from 205.234.175.175: bytes=32 time=26ms TTL=58
Reply from 205.234.175.175: bytes=32 time=23ms TTL=58
Reply from 205.234.175.175: bytes=32 time=25ms TTL=58
Reply from 205.234.175.175: bytes=32 time=24ms TTL=58
Ping statistics for 205.234.175.175:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 23ms, Maximum = 26ms, Average = 24ms

More impressive than that, it was only seven hops away from me. That includes my local router as the first hop, followed by four hops within my ISP's own network.

To get some confirmation of this result, I SSH'd into my Linux server hosted in a UK data center and repeated the test there. The difference was just as clear.

[jnthn@jnthn ~]$ ping programmersheaven.com
PING programmersheaven.com (68.142.114.70) 56(84) bytes of data.
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=1 ttl=108 time=161 ms
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=2 ttl=108 time=154 ms
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=3 ttl=108 time=162 ms
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=4 ttl=108 time=162 ms
--- programmersheaven.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2998ms
rtt min/avg/max/mdev = 154.588/160.262/162.414/3.281 ms
[jnthn@jnthn ~]$ ping pheaven.cachefly.net
PING pheaven.cachefly.net (205.234.175.175) 56(84) bytes of data.
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=1 ttl=57 time=12.1 ms
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=2 ttl=57 time=11.9 ms
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=3 ttl=57 time=12.2 ms
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=4 ttl=57 time=12.2 ms
--- pheaven.cachefly.net ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 11.932/12.130/12.227/0.178 ms

So clearly there are advantages for me as a UK user of Programmer's Heaven. But what about our US users? I have an account on a server in Texas, so tried it out from there.

[jnthn@naobi ~]# ping programmersheaven.com
PING programmersheaven.com (68.142.114.70) 56(84) bytes of data.
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=0 ttl=114 time=50.8 ms
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=1 ttl=114 time=45.7 ms
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=2 ttl=114 time=46.1 ms
64 bytes from programmersheaven.com (68.142.114.70): icmp_seq=3 ttl=114 time=45.8 ms
--- programmersheaven.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 45.774/47.161/50.842/2.145 ms, pipe 2
[jnthn@naobi ~]# ping pheaven.cachefly.net
PING pheaven.cachefly.net (205.234.175.175) 56(84) bytes of data.
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=0 ttl=54 time=34.5 ms
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=1 ttl=54 time=34.3 ms
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=2 ttl=54 time=34.3 ms
64 bytes from vip1.cdn.cachefly.net (205.234.175.175): icmp_seq=3 ttl=54 time=34.4 ms
--- pheaven.cachefly.net ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3002ms
rtt min/avg/max/mdev = 34.390/34.443/34.579/0.201 ms, pipe 2

Here, the location between the source and destination of the ping was much smaller, but still CacheFly gave a small advantage. It's not anything like as impressive as the difference it makes in the UK, but that's to be expected.

Deploying CacheFly

While many pages on Programmer's Heaven are delivered dynamically, there are many things that remain static over long periods of time. We chose to move many of the image files to the CacheFly servers. We then changed the links in one of our page templates to refer to the images on the CacheFly server rather than the Programmer's Heaven one.

The result of this is that the image files downloaded sooner, which decreases the overall load time the first time you visit Programmer's Heaven and any future time when the image files need to be re-fetched. While we have only rolled this out over some of the site, we plan to use it sitewide in the near future.

And that is the story of how we made Programmer's Heaven load faster. Enjoy!

---

Jonathan Worthington started programming at eight years old and hasn't stopped since! He holds a degree in Computer Science from the University of Cambridge. Specializing in programming languages and the tools that support them, Jonathan contributes to the Parrot virtual machine and the Perl 6 project. When he's not working, he loves to travel, especially to places with beautiful scenery.