Testing an Aspnet MVC app with Ruby

Why testing through the UI is hard(ly done)

Writing UI tests for a web application can be quite cumbersome and is a painstakingly repetitive task. Also, the UI of an application tends to change often during development, making UI tests a brittle resource. To top this, when running UI tests, you need to count on a browser to act as a host for them, making them a lot slower than regular unit tests.

It is for these and other reasons that people often don't go through the trouble of writing UI tests. And they are, in my opinion, right to do so.

But still, I have seen a lot of applications break in the UI after making minor changes. If these errors don't pop up in your staging environment, you are left with a red face once you deploy to production (and yup, I have seen this happening too often, to not take steps to fix this). And no, people don't always go through the entire application when it's on the staging environment. The times where I've seen full test plan execution for an application, before it is allowed to go through to production, can be counted on (less than) one finger. The only fallback we have here are our unit tests and as the name says, they are for testing (small) units, not entire applications (read: end-to-end testing).

There are of course testing frameworks you can use to run your UI tests, Selenium being one of them. I have been using Selenium on a couple of projects, but often give up quickly because of the slowness of the tests, because tests are hard to maintain, ... and so on.

Now, I recently followed the Ruby and Rails for n00bs session of my good friend Michel (@michelgrootjans) and got introduced to a couple of Ruby test gems that actually open up a couple of opportunities you can use in your .NET applications as well.

In this blog post I want to give you an overview of how you can set these Ruby test gems up, so they can run UI tests for an ASP .NET MVC application (or any .NET web application that is). The technologies (and gems) I will be describing here are:

• Ruby (of course)
• RSpec
• Capybara
• Selenium
• PhantomJS
• Poltergeist 

Setting up the test environment

First of all, let's get our environment setup. For this, you will need Ruby, which you can find here. Once installed, you can test if all went well by running the following command from the command line:

irb

This should open up a Ruby REPL loop where you can test out statements (for instance 1+1, should give you the answer of 2).

Once this is setup. Exit the REPL loop and start installing all the necessary gems:

gem install rspec

gem install capybara

gem install selenium-webdriver

gem install poltergeist

One other thing we now need to install is PhantomJS, which you can find here and Firefox (which is used by the default selenium web driver, I will give you other options further down this blog post). And that's all we need to get started.

Writing a first test with rspec

We can now start writing tests. Our tests will be placed in separate _spec.rb files in a spec folder. For instance if you want to write specification tests for customers, you will have a customers_spec.rb file, for products, you will have a products_spec.rb file. Of course you are free to use whatever naming and grouping of your specification tests, just make sure you have a spec folder with in it at least one _spec file.

A spec looks like this:

describe 'adding a new customer' do
  it 'should be able to create a new customer' do

  end
end

As you can see, you start of with a describe, which tells what kind of behavior you want to test. The it part contains the expected outcome (we will add this later). Don't worry about the Ruby syntax for now. For people using a BDD-style testing framework the describe and it syntax should look familiar.

The things we will want to test, are going to be the behavior of a website. For this we will use the capybara and selenium gems. And since we want this first spec and all following specs to be using the same setup for our tests (eg. url of the site we will be testing), we will use a spec_helper.rb file for this. In this file we will require all the necessary gems and do all of the setup. Each individual spec file will then require this one spec_helper file:

require 'capybara/rspec'
require 'selenium/webdriver'

Capybara.run_server = false
Capybara.default_driver = :selenium
Capybara.app_host = 'http://localhost/TheApplication'

After the require statements, we configure capybara. First of all, we tell it to not run its server. Since capybara is essentially a rails testing framework, it runs under a rack application by default. We won't have this, since our app will be running in IIS (Express) or something similar. Next we tell it which driver to use and last where our application is located (you could test www.google.com if you'd like).

Now that we have this setup, we can start writing a first test. Capybara actually allows you to click buttons and fill out fields in a web application, and that's what we will be doing:

require 'spec_helper'

describe 'adding a new customer', :type => :feature do
  it 'should be able to create a new customer' do
    visit '/Customer'

    click_link 'Add'

    fill_in('first_name', :with => 'Gitte')
    fill_in('last_name', :with => 'Vermeiren')
    fill_in('email', :with => 'mie@hier.be')

    click_button 'Create'

    page.should have_text 'New customer created'
  end
end

As you can see, capybara has methods for clicking links, filling out fields, choosing options, ... and in the end for testing whether you get an expected outcome in your application. You can find the capybara documentation with additional capabilities here.

Also notice I added the :type => :feature hash to the describe, so the test will be picked up as an rspec test.

Once you have this spec file, you can actually run your test. For this fire up a command prompt, cd to the directory just above your spec directory and run the rspec command:

rspec

You will notice this command will take a bit to start up, but once it's running, it will start up a firefox window and show you an error in the command prompt, since I assume you don't have anything implemented yet to get the above test to succeed. I leave it up to you, the reader, to write a small web app to get this test passing. Once you have this and you run the rspec command, you will see the different fields in your browser filled out with the values you specified in your test.

Improving our test

Now, I promised you, the return on investment with rspec, capybara, ruby, ... would be worth the effort. First of all, I can tell you from experience that the above capybara test is a lot cleaner than comparable tests written in .NET. But above that, we can do more. 

We can start of by having our tests run headless, meaning that we won't be needing a browser window anymore. For this we will use poltergeist and phantomjs. You will need to alter the spec_helper file for this:

require 'capybara/rspec'
require 'selenium/webdriver'
require 'capybara/poltergeist'

Capybara.run_server = false
Capybara.default_driver = :selenium
Capybara.javascript_driver = :poltergeist
Capybara.app_host = 'http://localhost/TheApplication'

Capybara.register_driver :poltergeist do |app|
  Capybara::Poltergeist::Driver.new(app,
                                    :phantomjs => 'C:\\some_path\\phantomjs.exe')
end

This adds the extra requirement for poltergeist and adds the javascript_driver configuration setting. Additionally we tell capybara where it can find the phantomjs process.

You will also need to add the :js => true hash to your describe statement for it to run in phantomjs:

describe 'adding a new customer', :type => :feature, :js => true  do

If you run your tests now with rspec, you will notice, firefox will not run, you will just get a summary in the command prompt of which tests succeeded or failed.

I ran some tests with our own web application and noticed that the headless tests ran twice as fast as the tests using the browser.

Further improvements

I am actually already quite happy with this first test setup. It's still not the fastest running tests, but it does allow me to get in some easy end-to-end testing, or to even start doing some BDD style testing with this. You can also use cucumber if you'd want to to instead off rspec. 

In my own setup I extended the above example with some extra configuration settings to be able to easily switch my testing environment from my local to the staging environment and from headless to browser testing. 

I also did some tests with chrome (doable) and IE (very, very slow test runs!), but for now prefer the firefox and headless setup. 

I would also like to add some extra stuff to setup and clear my database before and after each test. That is something I haven't figured out yet, but should be easily added. 

Doing the dynamic thing (Part I)

A while ago I was asked to give a presentation to co-workers about the new features in the 4.0 .Net framework. Being a bit of a language geek, this job, of course, was cut out for me. A lot of topics passed the revue, but the most interesting one, in my eyes, was the new dynamic keyword and the new dynamic features in version 4.0 of the framework.

However it's a bit sad that, when searching the web for good examples of dynamic programming, you get so few practical answers. Most sites stay on the surface of really discussing dynamics and how they can be employed in real life problems. Also, most blog posts about the topic were made 2 to 3 years ago. I'm guessing that means dynamics didn't really kick in as expected. Why, you would ask? In my opinion (and according to the reactions I've heard), people really don't like to live without their code completion for starters. They feel dynamics is a step back (towards plain old scripting languages aka VBScript, JavaScript, ... just to name a few). So in this, and next, post(s), let's look at dynamic programming, what it is, and how it can help solve some of our more practical issues in programming. Because, let's face it, a language like Ruby does the dynamic thing and is widely popular, there must be some revenue in employing dynamics, no?

First of, let's review a simple example of dynamic (one of the many found out there).

Dynamic can represent any type:

dynamic d1 = 7;
dynamic d2 = "a string";
dynamic d3 = System.DateTime.Today;
dynamic d4 = System.Diagnostics.Process.GetProcesses();

And I can quite easily assign these variables to their statically typed counterparts:

int i = d1;
string str = d2;
DateTime dt = d3;
System.Diagnostics.Process[] procs = d4;

So, I hear you think, that's like var and object. Indeed, like var and object, but with a different flavour. Because with var and object I can also do something like this:

object o1 = 7;
object o2 = "a string";

var v1 = 7;
var v2 = "a string";

Looks the same, right? Yes, but not quite. See, the thing is, object and var are still statically typed, meaning that at compile time their type is checked. The compiler knows o1 is of type integer and o2 is of type string, just as it knows for v1 and v2. The d1, d2, d3 and d4 variables, however, are dynamically typed, meaning their type is not checked at compile time, but at run time.

For a dynamic type I can do this:

int oKButCrashAtRunTime = d2;

At compile time I will get no errors whatsoever for this line of code. The compiler says it is valid.

Something I cannot do for object or var:

int notOkAtCompileTime = o2; //compile time error
int alsoNotOkAtCompileTime = v2; //compile time error

At run time however, the line of dynamic code will give me a big fat runtime exception. It's not because something is dynamic you can suddenly assign a string to an integer. So with dynamic types, you need to be extra careful.

These runtime exceptions are part of the reason why people don't like dynamic programming. You only know you've done bad once it crashes (in the user's face). But then why are there tons of programs written in dynamic languages like Ruby? Well, mainly, because Ruby programmers employ a slightly different way of writing code. And if you want the same confidence for your (partly) dynamic .Net code as Ruby programmers have for their (totally) dynamic Ruby code, you will have to learn some of those best practices. Those however, together with a practical .Net example, I will leave for a next post.

Also, dynamic programming brings in new possibilities when developing code. For starters, you get a much easier experience when working with COM objects (eg. in Excel, Word, but also objects from JavaScript). Another pro is the ability to add dynamic behaviour to your static programs. This as well I will leave for one of my next posts.