Open Source by FiveRuns

For the last few days, we’ve been running a survey to get some customer validation data on our pricing and features for Dash, and the response has been great. Since it’s only been a few days, I don’t want to bias anything by giving out interim results. Instead, I wanted to encourage Dash users to take the survey – it takes most people less than 1 minute.

So – please login to Dash and give us some feedback, it’ll be quick!

Also – a shout-out to the folks at SurveyGizmo: we met up with them at the FiveRuns, OtherInbox and Austin on Rails SXSWi 2009 party and now I can say from experience – it’s a great app. and I recommend using it.

dash-javascript is our latest little invention. It’s a bit different from dash-ruby, dash-python, et al.; those are about sending data to Dash. dash-javascript is about pulling data from Dash.

We’ve built dash-javascript because we know that Dash isn’t where everyone wants to see their data. Many apps have an admin backend, or even a custom dashboard pulling some ad-hoc metrics. There’s no reason those folks should have to log into Dash to get their numbers. We also know that ya’ll love showing off your feats of optimization with pretty graphs. Who are we to stand in your way?

So, dash-javascript gives you access to data from your Dash apps in whatever app you want. Suppose you’ve got something like this in your app’s administration backend:

<div id="health">
  CPU value: 
</div>

You can insert the latest CPU value for your app with this bit of jQuery-flavored JavaScript:

$('#health').dash({fetch: 'latest', token: 'your-read-token', metric: 'cpu'},
  function(value) {
    $(this).append(value);
  });

That’s it. You’re free to display data however you like. Throw it into a DOM element (as above), graph it with your favorite canvas or Flash-based charts or do whatever fits your taste.

Consider this an early-access preview. The API for this will change depending on what people need. We’re also planning on building easily embedded widgets, so if there’s any particular visualization you’d like to see, let us know!

You can get dash-javascript via GitHub; it’s even documented! If you have questions or feedback, we’d love to hear it.

Bradley Taylor (of RailsMachine) has come up with two new uses for Dash and Sensor plugins – both of which are great illustrations of the potential in the Dash service.

In Cucumber metrics with Dash: Bradley describes how he uses Dash to visualize the results of running Cucumber (i.e. passing, failing, skipped, and pending steps.).

instead of writing boring monitoring plugins from scratch, you can now do behavior driven ops! Transform from a grumpy, misanthropic sysadmin to a hipster, agile developer instantly.

Stay tuned for more examples & feel free to let us know how you’re using Dash (at support.fiveruns.com) and we will pass them on to the community.

One of the great things about Dash is its flexibility. This flexibility allows us to instrument app frameworks pretty quickly, and with the way our support for custom recipes is designed, it’s simple to mix-and-match metrics to get a better view of what’s going on with your application—in terms of its performance and business goals.

It was easy to get Dash working with Merb 1.0.x, and we’d like to announce our support for the framework, to include:

• Drop-in, simple installation similar to our Rails installation process
• Support for auto-adding our recipes for ActiveRecord or DataMapper (anyone want Sequel? Let us know!)
• Full-support for your own custom metric recipes and reports, as usual
• Identifying slow/frequent actions to target for performance fixes
• Smart exception capturing and notification via Twitter, Campfire, and Email
• Deployment tracking (with special support to link to GitHub to view the commit), notification via Twitter, Campfire, and Email

This is just the beginning.

Want in on the private Beta? Request an invitation.

As with the rest of our Dash instrumentation, the fiveruns-dash-merb gem is made available via GitHub under an MIT-style license. Community contributions are welcomed!

Welcome to the third installment of configuration-automation, a Ruby on Rails environment setup script. We’ll take a remote Linux server from first boot to fully configured, live Rails application server with just a hostname and password. This version includes a menu of Rails applications to choose from and replaces the nginx/mongrel servers with Phusion Passenger. Four popular Rails applications: Radiant, Spree, jobberRails and El Dorado are available to be deployed automatically.

What’s Needed

All you need is Capistrano and git installed on your *nix client to download and execute the scripts. On the remote server, Ubuntu 8.04 or 8.10 with ssh listening is supported, such as a Slicehost VM or VMWare image.

Using The Script

Just download the files from github and run:

git clone git://github.com/mmond/configuration-automation.git
cd ./configuration-automation
git checkout -b origin/production-v1
sh configure_ubuntu.sh

What Happens

An ssh connection to the Ubuntu server updates core system libraries and installs dependencies such as Ruby and MySQL. The Rails application chosen is deployed via Capistrano with a second ssh connection. Each connection requires the target server’s password or you can install an ssh key pair to skip those prompts.

To install additional Rails applications to the newly configured server, run configure_rails_apps.sh from the same ./configuration-automation directory. The main script can also be rerun from the same directory against other blank Ubuntu targets.

And Why?

With no DSL to learn or manifests to build, this is a fast, simple way to bootstrap a Rails server. I built it to test (and quickly retest) compatibility with FiveRuns Manage and TuneUp product updates. As such, it makes a decent acceptance test environment builder. It is also a way to experiment with a functioning Ruby on Rails app server without working through every installation issue.

It’s not intended to meet the meet the same needs as more powerful tools like Puppet, or now Chef, though you could easily add these after running configuration-automation. For EC2 environments, also check out Matt Conway’s cool Rubber Capistrano/Rails plugin.

The Rails app versions are frozen to today to ensure their project updates don’t break the automation. I’ll refresh these in later updates to configuration-automation. Suggestions and feedback are welcome via comments below or on my site.

Dash gathers and tracks your metrics. Some of those metrics might be easily accessible from your Ruby or Python applications but there’s a whole host of metrics that can’t be gathered directly from your application. That’s why we created Dash Sensor, a daemon with an extensible plugin architecture. We’ve built plugins for pulling metrics from nginx, apache, memcached and starling and use those to monitor our own Dash infrastructure. Here’s a few nginx numbers from our staging environment, e.g. 33 requests in the last minute:

Sensor is Ruby-based so the plugins are written in Ruby and they can do anything you can dream of: access a SOAP or REST web service, query a billing database, or process EDI documents from your System/360 mainframe. Fortunately for my own sanity, that last one is left as an exercise to the reader.

So go ahead and experiment with Sensor. The existing plugins are for common technical infrastructure like Apache and memcached but we believe the more interesting ones are the custom ones you can write to monitor and track your own business. Good luck and happy measuring!

One of the fundamental concepts in Dash is multi-platform support. Dash is not a Ruby library; it is an open specification with implementations in several languages.

The fact remains though that a programming language has a huge influence on the API design and supported functionality. Ruby’s powerful metaprogramming makes it quite simple for the dash-ruby implementation to wrap a method at runtime, just by declaring it:

recipe.counter :requests, "Requests", :method => 'ActionController::Base#perform_action'

This code tells Dash to look up the ActionController::Base class, find the perform_action method on it and dynamically wrap it in code which counts the number of invocations.

The Java API is more verbose:

public static class Requests extends BaseMetric {
    public String getName() { return "Requests"; }
    public IMetricCallback getCallback() {
        return new IMetricCallback() {
            public NamespaceValue[] getCurrentValue() {
                return basicValue(getRequestsAndReset());
            }
        };
    }
}

A metric definition is about 10 lines of code and that doesn’t even count the implementation of the getRequestsAndReset method. Essentially we are forcing the developer to instrument their code and the callback just gets the result of that instrumentation.

We’d like to simplify this API as much as possible by doing the instrumentation automatically. Since this API needs to work with all Java applications, we like to stick with the facilities in Java 5+ and not rely on Spring-AOP or similar. Can Java wrap arbitrary code at runtime with a minimum of modification to the target application? The closest thing we’ve found so far is Groovy’s dynamic MetaClass capabilities or AspectJ’s runtime weaving. Java pros, any other suggestions or ideas?

Today we are very excited to announce the private beta of our new service FiveRuns Dash. With Dash, we have a new take on monitoring & measuring. This is our first release of Dash outside of friends & family, and we are eager to get other people using it and giving us feedback.

In the past, we built Manage and TuneUp, both of which focus exclusively on measuring specific performance numbers. With Dash, we’ve gone about it differently: we built an extensible system to measure anything, and then used that to build implementations for specific languages and frameworks, initially Ruby & Rails, Python and Java. Beyond those initial uses, people are using it to measure the important technical and business metrics collected from their own applications.

Out of the box, we’ve got selected instrumentation and reports for several web application frameworks: Rails, Sinatra and some initial support for Django, with more to follow from the community and FIveRuns.

With this release, we support instrumentation and reports for Ruby, Python and Java applications / daemons as well. Additionally, because Dash at its core is a specification for collecting metrics, you can implement the Dash specification in any language to collect any business or technical data you want. Stay tuned, we will publish more information on our RESTful API soon.

You can use Dash to easily measure custom business and technical metrics from within your application by tweaking the Ruby/Python/Java plugin to specify additional metrics.

Another source of metrics lives outside your application. To handle this, we provide a small standalone Ruby app named Sensor. Initially, we’ve written plugins to gather metrics from memcached, apache, nginx and starling. You can easily implement your own plugins to Sensor to gather metrics from other sources. Look for more posts on Sensor in the days and weeks ahead.

Additionally Dash will let you:

• Track deployments and exceptions and sending notifications of those events via E-mail, Twitter and Campfire

• Share the application metrics and reports with friends. You can invite colleagues and friends to help you monitor your applications.

A big focus for us is the creation and sharing of metric recipes among the community – to let the community decide what is important to measure, to enable multiple opinions and perspectives to proliferate. Today, we deliver our instrumentation in recipes and describe how you can create recipes. As more people adopt this, we hope others will build their own recipes, challenge and extend our recipes and share them with the community. We look forward to the community joining us to enhance the capabilities for Django, Rails, Sinatra, Merb, etc. as well as extending the reach of Dash by adding new language support.

We’ve built Dash on top of what we’ve learned from Manage and TuneUp and we believe it’s a great step forward. However, as it’s new – we know there’s lots of room for improvement and we’re eager to get your feedback! Go ahead and sign-up for the private beta and we will get you in as soon as possible. We are here to answer any questions or listen to any comments you might have.

My wife recently begun the final push to finish her PhD dissertation. You care because this means I’ve had plenty of time in coffeeshops with her, working on open source projects.

First up, memcache-client. I’ve taken over the project from Eric Hodel and put in some significant changes since 1.5.0. Read the gory details at my blog. We’ve been using this version internally here for our upcoming Dash product for the last two weeks, against both memcached and Starling, with good results so far. It does re-introduce a native extension which breaks the gem installation with JRuby. I’d appreciate any advice JRuby people can give me on how to play nicely in their world; I’d like to support JRuby by providing pure Ruby fallback code but don’t know if that’s possible.

Next, data_fabric. Since the official release of 1.2 with full support for ActiveRecord 2.2, we’ve put out two patch releases to fix issues we’ve found. Like memcache-client, we’re using it in production with good results.

Finally, I just pushed a forked version of Rails’ well loved deadlock_retry plugin. This version adds innodb status logging and the ability to run as a gem, rather than a plugin. If you are using MySQL and have seen deadlock in your applications, you’ll want to take a look at this.

We hope you find these useful; let us know how they work out for you!

We congratulate the Merb team on a successful 1.0 release!

We’d like to announce a new version of the TuneUp panel slice gem for Merb (which sits up at GitHub), as well as the core TuneUp dependency.

v0.5.3 is a bugfix release, addressing issues with uploading runs and removing development dependencies that were causing loading issues. The installation instructions have also been updated, removing the now-unnecessary installation task (assets are now copied automatically on startup, as with the TuneUp for Rails) and changing the routing instructions.

Many thanks to Matt Aimonetti for his continued help tracking down issues and providing Merb slice “best practice” guidance. Cheers!

Last year, we launched a partnership with BitRock, an innovator in open source tools and stacks, and introduced FiveRuns Install, a Ruby on Rails stack optimized for production environments. Install has been a great success, with thousands of downloads from the community. This week, we’ve happy to announce that we’ve expanded our partnership with BitRock and now their BitNami RubyStack, a freely available, ready to run distribution of Ruby on Rails, will include FiveRuns TuneUp, our free/open source application profiling tool for Ruby on Rails and Merb developers.

The BitNami RubyStack was released in 2007 and has been downloaded tens of thousands of times. It’s a freely available, ready to run distribution of Ruby on Rails, packaged with BitRock’s multiplatform installer, making it easy to have an enterprise-ready Rails environment up and running in just a few minutes. By including FiveRuns TuneUp with the RubyStack, we wanted to give developers a complete environment for installing, developing, and profiling Ruby on Rails applications that can be optimized for performance and scale.

BitRock has been a terrific partner, and we’ve been continually impressed by their expertise and leadership in the open source stack business, so we’re very excited to further our relationship and offer up this bundle to the Rails development community. RubyStack is currently available for Linux, Mac OS X, and Windows, with support for Solaris coming soon. It is free to download and use under the terms of the Apache 2.0 license.

For more details, or to download a copy, visit BitNami.org.

TuneUp is FiveRuns’ open source plugin for profiling the performance your Rails and Merb applications. TuneUp is intended to be run locally in your development environment, since the instrumentation returns detailed analysis of a single browser invocation. Bottlenecks can be identified before deployment and before your users experience a performance issue.

Why?

There are some times it may be useful to run TuneUp in production. Perhaps you’d like to share the profiling view across a team. Maybe you’d like to compare the metrics of that same request, against your production database’s configuration, on the actual hardware or under the live platform’s operating system.

All you need to do to activate the TuneUp instrumentation in production is create the following file:

 /YOUR/APP/config/tuneup.rb  

Containing the following:

Fiveruns::Tuneup.config do |config|
    config.environments << :production
end

That’s it. Restart your rails app on the server and it will render the TuneUp instrumentation bar to your local browser. From there, you can toggle it on and off and profile the live server requests remotely.

Why not?

Running TuneUp in production should be done in a closed environment. The instrumentation bar, sequel queries, controller names and other analytics about the code base are displayed. Also, multiple browsers accessing the application concurrently may skew the profiling results.

If you’d like to permanently monitor your servers’ Rails performance in production, check out FiveRuns Manage.

I’ve been playing with benchmarking tools this week. After reviewing the open source options, I ended up working with just a few of the dozens of choices. Some are new and Rails specific. Some are no longer maintained yet remain popular choices. Here’s a sample:

• httperf
• Siege
• RailsBench
• ApacheBench
• Selenium
• Rails performance test generator

There are not only a lot of tools, there are a variety of approaches to take. Are you looking know how fast your application can return a single user’s transaction? Is a better durability test to know how much traffic your application can sustain over longer periods? Should you test individual pages of the web app or build workflow sequences to more closely simulate real world activity?

For your performance test method, what’s your choice of benchmarking tools?

So your shiny new Rails application is ready for production. Time to configure the server application layer. A common question we hear at FiveRuns is: What’s the best number of Mongrels to use? There are a plenty of opinions, including yours, so feel free to comment below.

Test Drive

Early on, Zed recommended performance testing with httperf and an increasing numbers of mongrels to determine the best number for your setup. Test a file served directly from the web server and then test your app via a single mongrel instance. This provides a static file reality check and then a decent baseline from which to improve the responsiveness based on additional mongrels.

> 1

Depending on the memory footprint of each mongrel process and the available system memory, you may have an upper limit to consider for your server. Some administrators use at least two mongrel processes to balance the queue and avoid any locking behind a single mongrel or Rails request. My tests with a very entry level 256 MB virtual machine yielded the best performance using 3 mongrel instances behind an Apache2 front end.

Us

The average number of mongrel processes used by our customers is 5 per Rails app and 12 per server on systems with 8 GB of physical memory. This is what we run here at FiveRuns too. For example, a common setup is to have two high volume applications, served by 5 mongrels each and a lower volume Rails app with 2 mongrels. A lightweight front end like nginx balances traffic among the mongrel ports in use and optionally, any additional application servers in the hardware pool.

None?

Choosing not to answer the “How many Mongrels?” question is becoming a popular option. Passenger handles the port management between the front end and application layer, including the optimal number of back end processes. Thin is lightweight and crazy easy to install and configure. Each yield comparable or better performance metrics. Ebb also lists significant performance gains over mongrel.

So with configuration ease and performance gains, why choose mongrel today? Still grateful for delivery from FastCGI/lighttpd? (like me?) Perhaps you are gauging the community adoption of the alternatives? Maybe it’s a proven part of your existing architecture?

Welcome to the second FiveRuns blog post to automate server configuration for Ruby on Rails. We received great feedback and suggestions about the original script, Automatic Rails at Slicehost, which installed the basics. This time we’ll use standards like Capistrano deployment, separate web and application layers and an active database backend. While the Hello World application surely dazzled your friends and family, this time we’ll take it up a notch and deploy a working, full stack community web application, Trevor Turk’s cool Rails app, Eldorado.

The What

This script connects to a new Ubuntu 8.04 server and configures all necessary applications and libraries. Since we are installing a lot more this time, it’s a bit long to post here, so I’ve linked to it on Github. You still only need to provide 2 pieces of information: the target address of your server and the password. Execute the script locally which downloads Eldorado, the associated configuration files and then connects to your server to push the installation.

• Ruby 1.8.6
  
• Rubygems 1.2
  
• Sqlite3
  
• MySQL 5.0.51a
  
• Thin 0.8.2
  
• Hello World example Rails application
  

• nginx 0.5.33
• Mongrel 1.1.5
• Eldorado Rails application
• Capistrano deployment
• Rubygems 1.3

The How

1. Download the script from Github. I’ve abbreviated the URL.

wget http://tinyurl.com/57sjce -O configure_ubuntu_eldorado.sh

2. Run the script.

sh configure_ubuntu_eldorado.sh

That’s it! You’ll be asked for your password twice, once to set up the OS via ssh and once to run the Capistrano deployment tasks. Speaking of passwords, best practices include dedicated deployment and MySQL user accounts. This script uses the root account for simplicity.

The Why

The script originated as a DRY effort for Slicehost deployments. You can now point it at other providers, remote Ubuntu servers or virtual machines. It’s become a handy way to explore a working Rails environment without manually installing every component. Also, writing configuration automation is a great way to hone your deployment skills. Thanks go to Trotter Cashion for the suggestion to maintain the scripts on Github, where they’ve been forked and extended. Check out Jørgen Orehøj Erichsen’s blog adding support for Passenger and Ruby Enterprise Edition.

All the scripts and configuration files are on Github so feel free to post any feedback or suggestions.