Archive

Posts Tagged ‘Distributed Caching’

Pre-loading the distributed cache

January 22, 2010 Leave a comment

We have been using memcache for distributed caching in one of the products that we were working with. We had 4 application servers with 2GB of memory dedicated to memcache in each of the machines. A lot of times when we delivered new functionality or a patch, we needed to restart the memcache servers. And then for a day, the site thrashed. We got horrible response times for a lot of our pages which were supposed to be cached. We digged a little deep and found out that the cache loader which was supposed to pre-load the cache needed about 50 hours to complete. No wonder we were getting horrible response times in pages that were supposed to be cached. What we actually needed was a multi-threaded pre-loader running on two different machines. With some code optimizations and 16 threads pre-loading the pages into memcache, it took us about 3 hours.

Follow

Get every new post delivered to your Inbox.