Do you have experience with squid or varnish proxy servers?

If any of you readers out there are experienced sysadmin/webmaster types I could use a little help. Every time this site gets on the front page of Digg or Slashdot it has the tendency to go down, or at least go very slow. I’m running wp-super cache and I have a dedicated server, but there are a lot of images on the site (we serve about 700gigs a month), and everyone of them has to spin up a 24 meg apache process. I would like to set up a caching proxy using either squid or varnish to handle all of the static content. However my site has a pretty odd vhosts setup generated by the control panel software my host uses and I’m not really sure what i need to do.

So how about it, anyone out there who would be willing and able to help me get it set up?

6 comments
  1. Eric says: March 8, 200812:18 pm

    Perhaps I am misunderstanding, but you want a proxy server to cache -dynamic- content. Serving static content that way isn’t going to do much. There is also a lot you can do to trim Apache’s footprint — 24mb is a lot. Also much you can do to optimize the Apache config — obviously if you are handling only one request per process you are in big trouble (this must be wrong). Beyond that, consider using something like lighttpd (lighttpd.net) to serve images and static content.
    A proxy is a good way to do brute force caching if you have no other alternative, but it’s complex and has many drawbacks.

  2. Charlie says: March 8, 20089:01 pm

    Well I have wordpress caching dynamic content, so it’s not generating that that’s the problem. It’s just the number of requests coming in that runs me out of memory. I was thinking of using varnish as you suggested for lighttpd, it would sit in front of apache and handle all of the static (images, css etc) content, then pass the rest through.

    I don’t really know much about apache config to know how to turn that stuff off. Do you have any suggestions on how to do that?

  3. Eric says: March 10, 200812:07 pm

    The WP cache is not that good, relatively speaking — you are not caching everything. But the question is if you are CPU or memory-bound. From what you say, the latter is true. If you are unsure, you can easily verify this by running the “top” command while the server is displaying the symptoms and noting the available memory and swap usage, as well as the load average. I imagine you’ve done this already, of course.

    What you are proposing is effectively the same thing as using lighttpd. Likely both would provide similar performance although some tests I’ve seen suggest Varnish outperforms lighttpd on really small files, but not necessarily on larger ones. I have not used Varnish myself.

    Of course one can’t say definitively without knowing the specifics, but strictly speaking using a web cache/reverse proxy like Varnish only for static content is not on it’s own the most efficient thing. Rarely have I seen a situation where a high traffic load doesn’t tax memory -and- cpu and WordPress in particular is very CPU intensive. You’ll just have to test it — of course the advantage to using Varnish from the start is that you can then have it cache dynamic content as well, if required. And perhaps you have CPU to spare, so you just need to free up enough memory to prevent swapping.

    You can trim Apache by commenting out some of the modules in httpd.conf, assuming (as they are on Fedora) most are compiled to be loaded dynamically (DSO). Usually by default they are almost all turned on (for example, mod_ldap). It’s worth playing with this to see if you can get a little gain, but obviously it’s not going to be as good as simply not calling up Apache processes in the first place.

    If you haven’t already done so, check the MaxRequestsPerChild setting. It is “0″ by default on many distros, which -usually- isn’t appropriate. There’s a trade off there between efficiency of killing/spawning processes and reclaiming memory. The related settings of MaxClients, ServerLimit, etc are all key, and need to be tuned for your specific situation. I also find that lowering the Timeout is pretty key for getting rid of slow clients under high load. You can find guides on how to tune Apache, but it’s not nec. that easy.

    So back to your original question — assuming Varnish is the best answer for you, where do you need help? People generally set up Varnish to run on port 80, with Apache moved to port 81. Varnish then handles all incoming requests, passing off selected ones to Apache. Using Lighttpd, typically you have Apache serving normally then farm out selected requests (like images) to lightttpd serving on another port or address. Your vhost configuration in Apache isn’t really an issue in either case except maybe if you have a complicated port arrangement.

    Hope this helps in some way.

  4. Simon says: March 28, 20083:19 pm

    This probably comes a little late, but I’ll throw in my two cents. Having used squid for several years as an http accelerator, recently migrating to varnish cache, I can recommend both for caching of image and dynamic content. Simple ttl caching on dynamic content, even if it is only for 60 seconds, might save you when you are being slashdotted. Varnish is easier to configure, has rules flexible enough to handle our needs, and has proven itself capable of handling much higher load (sustained 1000 req/sec) than our previous squid installation (squid gurus might be able to squeeze more performance out of squid, but I couldn’t).

    Varnish is, as pointed out earlier, just part of the performance equation, but I consider it essential for situations that result in an extremely high number of requests for a relatively small number of objects (images, pages, etc.), such as a slashdotting.

  5. Charlie says: March 28, 20083:24 pm

    Simon » Thanks for the advice. I’ve actually got things running pretty smoothly now, though I’m sure apache is using way more ram than it should. We survived being on the front pages of digg, delicious, reddit and slashdot this week, though not simultaneously. I’m planning on switching hosting companies soon (mine has been having too many network problems and I don’t like their support) and when I do, I’m going to go for a clean install, with no cpanel gunk in the way. I’ll have to learn a lot more about setting up a server, but then I’ll be able to do it right. My current plan is to go with either varnish/apache or just straight nginx. I’m not sure yet.

  6. Rodrigo says: November 30, 20094:06 pm

    I have experienced with Varnish if you still need any help :) I had the same problems that you had

Submit comment

You must be logged in to post a comment.