In order to scale our service, a large portion of our Manage service runs asynchronously from the http://manage.fiveruns.com website. We use Starling, the fantastic queuing system behind Twitter, to manage jobs to be processed. One interesting twist to Starling is that it uses the memcached client API so any memcached client library can access Starling without any extra effort.
Since our service is 24/7, we needed fault tolerance, high availability and other such high-falootin’ ideals. Our plan was to have two Starling servers, one on each app server, with the job processors using the local Starling server on their own machine by default, falling back to the server on the other machine if the local Staring daemon died.
Seattle.rb’s memcache-client 1.5.0 library supports multiple servers and does make an effort to switch between them but it has three known problems:
- set() fails when the socket has been disconnected (via Twitter)
- it does not retry if a socket operation fails (via Will Bryant)
- it does not retry the current operation on another server if a server dies
The latter issue means that data will be lost when a server dies. We’ve fixed all three issues in our Github repository for memcache-client and Zachary Pinter is working on getting our updated code pushed into Edge Rails. Hopefully we’ll see an updated memcached client in Rails soon. In the mean time, please feel free to fork or clone our repository and use it in your own project!
Continued Discussion
9 responses to this entry
I’ve actually had really good results with Evan Weaver’s memcache client: http://github.com/fauna/memcached/tree/master
What’s the advantage of memcache-client over (in my experience) faster lib?
on June 12, 2008 at 10:30 AM
The only advantage appears to be one of marketing. When trying to determine what the “standard” Ruby memcached client is, memcache-client kept coming up. Evan’s client certainly looks like an attractive option. Thanks for the pointer, Jeremy.
on June 18, 2008 at 03:02 PM
It looks like you forgot to update the README doc for your fork:
“http://github.com/advany/starling/tree/master”
as well as the install directions:
sudo gem install advany-starling
on June 18, 2008 at 05:10 PM
Are there any resources on how to get fiveruns-memcache-client up and running? It seems like the instructions say do $sudo gem install fiveruns-memcache-client, but this generate all sorts of gem not found hiccups.
on July 08, 2008 at 08:19 AM
The gem name was set incorrectly. If you do a gem install now, it should work fine.
on July 09, 2008 at 03:54 PM
Hey. Nice work. I found that I had to remove the memcache client from the Rails vendor in order to load the gem. Don’t know if there’s a nicer way to do this such that I don’t have to edit the Rails version used.
Any luck getting the fixes pushed to the version that ships with Rails? Does that version even do the CRC32 natively?
on September 09, 2008 at 06:48 AM
Where can I ask questions about getting this to work? I’m tearing my hair out.
on September 29, 2008 at 10:32 AM
Brian: We are not aware of any mailing lists specific to Starling. Maybe you can find help in blog posts about it, like this Ruby Inside article or this Rubypond article, for example.
on September 30, 2008 at 02:52 PM
Thanks for responding. I’ll check them out.
on October 02, 2008 at 10:45 AM