Today, we are starting our very first series of Q&A sessions. In this session, we talk to Per Buer, the founder of Varnish Software. The topic of discussion is Comcast and Apple’s decision to use Apache Traffic Server (ATS) to power its CDN, instead of Varnish Software. Comcast has done the CDN industry a lot of good in going public with its analysis on why they chose ATS. In case anyone is wondering, Comcast and Apple decided on the Apache Traffic Server to serve as its caching platform. ATS is an open source software platform that competes with Varnish Software.
For me personally, the decision was a little surprising, being that Varnish has an actual company behind the caching platform. There is nothing like having the founding company back their popular open source software through the commercial side of the business, in that there is accountability when things go haywire. With the Apache Traffic Server, companies deploying the software are left at the mercy of the open source communities for fixes, updates, documentation, and so on.
Per Buer, thanks very much for your time. The public has heard one side of the story from Comcast on their decision to go with ATS, and why they selected ATS. Now we get to hear it from the other side. Let the session begin.
First Question
Is Varnish Software a better fit for a CDN than Apache Traffic Server? Why?
ATS is a pretty solid piece of software and it is pretty well suited for the CDN. Both TS and Varnish have good performance, usually enough to saturate gigabit link, which is what most people need.
There are CDNs based on both TS and Varnish, with the beforementioned Apple and Comcast ones probably being the largest installations. A very prominent Varnish based CDN would be Fastly, which has built a CDN that really highlights the reasons for using Varnish as a CDN engine. Their focus on performance and flexibility is what sets their CDN apart.
The reasons for using Varnish in a CDN is first and foremost the flexibility of the VCL language, gives the CDN a lot of power in how the content being cached can be treated. The flexibility is creating cache policies is more or less endless. To use CDN lingo, Varnish has a “rules engine” built into it that allows for great flexbility.
There are currently advantages that ATS has over Varnish. It will probably deal with a large dataset (think VOD) better than the built in storage engines in Varnish can. These are issues we are addressing and we’ll release a version of Varnish specifically crafted for CDNs later this year. And at this stage, it looks very promising
2nd Question
Does varnish provide all the bells and whistles needed to power a CDN, like SSL, SPDY, Gzip compression, security, query string caching, TTL, raw log reporting, and so on?
Most of these are well covered with a few exceptions. I’ll go over the ones you mentioned in order to give you an idea.
SSL
Varnish does not support SSL so you need to terminate the SSL connections before the requests hit Varnish. There are quite a few proxies that do that today and as far as we know there is no significant performance gain by having SSL built into the software itself. SSL is pretty complex and the lack of quality in libraries like OpenSSL has shown that not embedding it into Varnish has been a good decision. So far, most customers tend to agree.
SPDY
SPDY is not supported either. Currently we’re waiting for the IETF to ratify HTTP 2.0. When HTTP 2.0 is done support will be added. SPDY is too much of a moving target for us to implement at this point in time and the performance benefits are not obvious.
Gzip
Gzip compression is there and the implementation is rock solid. I believe Varnish has the only working implementation of Gzip that works well with Edge Side Includes (ESI).
Security
Security is very well taken care of. There are additions to Varnish that can make Varnish act as web application firewall so you’ll have the option of actually protecting the customers. In addition I might add that our security track record is excellent and we’ve had very few vulnerabilities since we launched the project.
Query String Caching
Query string caching is there. There are also modules available that can help out by normalizing the query string before caching so you don’t get duplicates when the parameter order varies.
The VCL Language gives you very good ways of manipulating TTLs.
Loggins options are plentiful. I believe Varnish is the only proxy where the logging doesn’t slow down the content delivery. Logging into various stats tools can happen without having the server log to disk first.
3rd Question
If a CDN implements Varnish Software, are there any missing pieces that require a CDN to look elsewhere? What are those missing pieces?
I assume you mean that the CDN uses Varnish Plus here. Varnish Plus adds a few more features, helps with cache invalidation, configuration management and real time reporting. If you are using the open source software you need to find replacement components for these as well. In addition you of course need to deal with these functions:
- You’ll need something to manage the configuration, what backends and virtual hosts are tied together, etc.
- Billing
4th Question
Comcast is using Apache Traffic Server for its caching platform, however, it had to stitch together many different software products to build a complete CDN platform, like Splunk for reporting, Tomcat, and so on. What do you think about their approach and is the Comcast platform scalable?
I think their approach is very good. Far too many CDN solution vendors come with a tightly, vertically integrated stack that makes integration a royal pain. With a component based approach you have the option of customizing the various bits and pieces into something that fits your needs. These tightly integrated CDNs also suffer from a fixed feature set. If they don’t support it out of the box it is more or less impossible to do and it usually takes five years if you are big enough for the vendor to grant you your wish. With loosely integrated software with open protocols we’re talking about something that is a lot more flexible and doesn’t require you to build your CDN business around the way the software is built. Because requirements to be competitive will change over time.
It comes with a price. You need to understand your infrastructure fully. But I believe it will be harder and harder not to do this. If you want performance you need to understand the how your CDN is operating and having some big, black box from a vendor won’t give you the options to tune and tweak it for performance or meet future challenges.
5th Question
Say I’m building a CDN, and have 10 POPs in various places around the world, with each POP having 50 servers? What steps would I need to take to create a CDN using Varnish? How would I install it, what components would I load first, what features get activated first, what about load balancing, and is their anything else you can bring up would be helpful? The goal here is to provide an upstart CDN an idea of what is involved in building a scalable CDN. Please include a diagram if it makes it easier?
The approach would be more or less identical for both ATS and Varnish. There are some minor differences with how configuration and logs are handled but overall they are very much alike.
You start out with your network. Get it up and running and get the capacity you need. Then you need to handle to traffic management, how do you identify the POP closest to the user and direct the user to the POP. Then configuration management, billing, logging (both for billing and debugging purposes).
6th Question
Varnish has a Professional Services group? What services do you provide in helping customers like Comcast implement Varnish software?
Yes, we do. Usually the people that hire us have us do pretty narrow stuff, like tuning Varnish or building extensions. When deploying a CDN we’re usually confined to our little box and deal with caching, having other taking care of the other components of the CDN.
As CDNs are becoming more and more complex, requiring more and more components to deliver what’s an acceptable service, the work that is “not the cache engine” takes up more and more time, so our involvement isn’t what it used to be. This is expected to change when we release the CDN version of Varnish.