Month: March 2011

Data driven tests

March 22, 2011

Technical

data driven tests, test, testing

I’m not sure if anybody uses the terminology “data driven test” but if you explain what it is, experienced people will tel you that they are bad. Data driven tests are tests with the same code repeating over many different pieces of data.

Let’s show an example. For my startup project Keep on Posting, I have a method that turns a blog url into a feed url. That method is critical for my application and there are many things that can go wrong, so I test it by querying a sample of real blogs. The test would be something like this (this is in Ruby):

[sourcecode lang=”ruby”]
class BlogToolsTest
BLOGS_AND_FEES =>
"http://blog.sandrafernandez.eu" => "http://blog.sandrafernandez.eu/feed/",
"http://www.lejanooriente.com" => "http://www.lejanooriente.com/feed/",
"http://pupeno.com" => "http://pupeno.com/feed/",
"http://www.avc.com/a_vc" => "http://feeds.feedburner.com/avc",
}

def test_blog_to_feed_url
BLOGS_AND_FEEDS.each_pair do |blog_url, feed_url|
assert_true feed_url == BlogTools.blog_to_feed(blog_url)
end
end
end
[/sourcecode]

Note: I’m using assert_true instead of assert_equal to make a point; these kind of tests tend to user assert_true.

The problem with that is that eventually it’ll fail and it’ll say something like:

[sourcecode]
false is not true
[/sourcecode]

Oh! so useful. Let’s see at least where the error is happening… and obviously it’ll point to this line:

[sourcecode lang=”ruby”]
assert_true feed_url == BlogTools.blog_to_feed(blog_url)
[/sourcecode]

which is almost as useless as the failure message. That’s the problem with data drive tests. You might be tempted to do this an re-run the tests:

[sourcecode lang=”ruby”]
def test_blog_to_feed_url
BLOGS_AND_FEEDS.each_pair do |blog_url, feed_url|
puts blog_url
puts feed_url
assert_true feed_url == BlogTools.blog_to_feed(blog_url)
end
end
[/sourcecode]

but if your tests take hours to run, like the ones I often found while working at Google, then you are wasting time. Writing good error messages ahead of time help:

[sourcecode lang=”ruby”]
def test_blog_to_feed_url
BLOGS_AND_FEEDS.each_pair do |blog_url, feed_url|
assert_true feed_url == BlogTools.blog_to_feed(blog_url), "#{blog_url} should have returned the feed #{feed_url}"
end
end
[/sourcecode]

and if half your cases fail and the whole suit takes an hour to run and you have 1000 data sets you’ll spend hours staring at your monitor fixing one test every now and then, because as soon as one case fails, the execution of the tests is halted. If you are coding in a language like Java, that’s as far as you can take it.

With Ruby you can push the boundaries and write it this way (thanks to executable class bodies):

[sourcecode lang=”ruby”]
class BlogToolsTest
BLOGS_AND_FEES =>
"http://blog.sandrafernandez.eu" => "http://blog.sandrafernandez.eu/feed/",
"http://www.lejanooriente.com" => "http://www.lejanooriente.com/feed/",
"http://pupeno.com" => "http://pupeno.com/feed/",
"http://www.avc.com/a_vc" => "http://feeds.feedburner.com/avc",
}

BLOGS_AND_FEEDS.each_pair do |blog_url, feed_url|
define_method "test_#{blog_url}_#{feed_url}" do
assert_true feed_url == BlogTools.blog_to_feed(blog_url), "#{blog_url} should have returned the feed #{feed_url}"
end
end
end
[/sourcecode]

That will generate one method per item of data, even if one fails, the rest will be executed as they are separate isolated tests. They will also be executed in a potential random order so you don’t have tests depending on tests and even if you don’t get a nice error message, you’ll know which piece of data is the problematic through the method name.

Note: that actually doesn’t work because blog_url and feed_url have characters that are not valid method names, they should be replaced, but I wanted to keep the example concise.

Since I’m using shoulda, my resulting code looks like this:

[sourcecode lang=”ruby”]
class BlogToolsTest
BLOGS_AND_FEES =>
"http://blog.sandrafernandez.eu" => "http://blog.sandrafernandez.eu/feed/",
"http://www.lejanooriente.com" => "http://www.lejanooriente.com/feed/",
"http://pupeno.com" => "http://pupeno.com/feed/",
"http://www.avc.com/a_vc" => "http://feeds.feedburner.com/avc",
}

BLOGS_AND_FEEDS.each_pair do |blog_url, feed_url|
should "turn blog #{blog_url} into feed #{feed_url}" do
assert_equal feed_url, BlogTools.blog_to_feed(blog_url), "#{blog_url} did not resolve to the feed #{feed_url}"
end
end
end
[/sourcecode]

and running them in RubyMine looks like this:
Startup Weekend Lausanne 2011

March 21, 2011

Personal

Eneo, eneo.me, entrepreneurship, Lausanne, Startup Weekend, startups, swlau

The Eneo team

This past weekend I participated in the Startup Weekend Lausanne 2011, my first startup weekend ever. It was amazingly interesting. I worked on an application called Eneo. Eneo allows you to talk with people around you using your mobile phone, people that you don’t know, so the next time you are stuck at the airport waiting in a 3 hours delayed flight, you don’t have to eat alone. Find someone else in the same situation and go for lunch.

I think the application has great potential and I was told that the jury didn’t understand it. I don’t know, they spoke in french. Supposedly the event was going to be in french and english but was all in french… I was bored when I wasn’t coding.

I’m thankful I didn’t present an idea, I would have done a poor job of doing it. I now wrote a list of things to do when presenting an idea. I was going to publish it, but it’s gold so I’ll keep it to myself, at least, until I use the techniques. Muahahaha.

I chose a company to work with, but the guy with the idea decided not to work on it because he didn’t get enough people. It was him, one designer and me, a programmer. I was surprised. Then I saw most groups were more than 5 people, some may have been even 8. For me that’s absolutely crazy. The startup sizes that make sense for me are 2, 3 and 4. Any less and it gets lonely (tell me about it!), any more and it gets crowded.

When the group I was in disolved, I turned into a free developer and the four or five groups that didn’t get a developer already started to pitch to me (now they did speak in English ;), at the same time. It was crazy. I think I got a glimpse of how investors feel. At any rate, immediate ego boost for a developer. If you are a developer I recommend to attend this events, is great for you and great for the event.

While working on our project I did make some mistakes. One was working on a mobile application. I haven’t done mobile in a while, I’m not fast at it; not weekend-fast at least. Give me a week and I’ll be rocking your phone, but in a weekend, no. I decided I wasn’t going to code, but I’m a coder and in less than two hours after joining Eneo I was writing code. I paid a step price when I tried to make the application look mobile by using jqtouch. I never used jqtouch before so there were a lot of things not working and I wasted precious hours on it. Big mistake.

My second mistake was trying to make a product. I wanted to woo the jury by showing an active product with users already. I failed to do that. I should have gone straight to coding a demo, not a finished product. What we ended up with was half product half demo, not good at either.

Here’s the app running in a desktop browser (imagine it’s a phone):

Next time I’ll do better.

When I need to run something periodically on production, I always implement it as a rake tasks and install it as a cron job. Nevertheless there’s some setup to do in the task to have proper logging and error reporting.

This is the template I use for creating those tasks:

namespace :projectx do
  desc "Do something"
  task :something => :environment do
    if Rails.env.development?
      # Log to stdout.
      logger = Logger.new(STDOUT)
      logger.level = Logger::INFO # DEBUG to see queries
      ActiveRecord::Base.logger = logger
      ActionMailer::Base.logger = logger
      ActionController::Base.logger = logger
    else
      logger = ActiveRecord::Base.logger
    end

    begin
      logger.info "Doing something"
    rescue Exception => e
      HoptoadNotifier.notify(e)
      raise e
    end
  end
end

While in development mode, it outputs to the console for convenience.

More sense of humor

March 4, 2011

Personal

fail, humor, sense of humor, Tumblr

If you are going to fail, fail funnily, like Tumblr…

Month: March 2011

Data driven tests

Startup Weekend Lausanne 2011

Rake tasks for production

More sense of humor