Category: Technical

Posts related to programming or other highly technical stuff that can only be of interest to computer geeks.

  • During the initial phase of development of a Rails application I don’t use migrations as migrations but as table definitions. Until I deploy I feel free to modify the migration files as much as I want and I have one per table.

    The downside of that is that the only way to apply the changes is to destroy all the tables and re-build them. I’ve explained how to do that in my post really resetting the database. The nice side effect of doing this is that you end up with a task that sets sample data to work with.

    Being able to quickly set up sample data or download production data is very important. It helps new developers getting started with the project but it also allows you to play much more freely with the project, do destructive actions and then in a quick command have system reset to a known state. Once you have sample data you’ll probably become as addictive as I am to reseting.

    But the truth is that 90% of the time you reset your data, you don’t need to nuke the database and re-create all records, you just need to delete all records and this is the code I use to do that:

    def destroy_data
      puts "==  Data: Destroying all data ".ljust(79, "=")
      sql = ActiveRecord::Base.connection()
    
      sql.execute "SET autocommit=0"
      sql.begin_db_transaction
    
      if sql.adapter_name == "MySQL"
        sql.execute("/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */")
        sql.execute("/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */")
      end
    
      tables = sql.tables - ["schema_migrations"]
    
      tables.each do |table|
        puts "-- Deleting all for #{table}."
        # So far, disabling and enabling keys was not needed.
        #sql.execute("/*!40000 ALTER TABLE `#{table}` DISABLE KEYS */") if sql.adapter_name == "MySQL"
        record_count = sql.delete("DELETE FROM `#{table}`")
        #sql.execute("/*!40000 ALTER TABLE `#{table}` ENABLE KEYS */") if sql.adapter_name == "MySQL"
        puts "   -> done: #{record_count} reconds"
      end
    
      if sql.adapter_name == "MySQL"
        sql.execute("/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */")
        sql.execute("/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */")
      end
    
      sql.commit_db_transaction
    
      puts "==  Data: Destroying all data (done) ".ljust(79, "=") + "\n\n"
    end
    

    Note that I’m not deleting anything in the table schema_migration. The output is more or less like this:

    ==  Data: Destroying all data =================================================
    -- Deleting all for blogs.
       -> done: 4 reconds
    -- Deleting all for posts.
       -> done: 10 reconds
    -- Deleting all for users.
       -> done: 11 reconds
    ==  Data: Destroying all data (done) ==========================================
    

    I also have some nice code to generate sample data, but that is for another post.

  • If you are getting this error:

    ActionView::Template::Error: undefined method `authenticate?' for nil:NilClass
    

    in your call to Devise’s user_signed_in? or similar, you probably forgot to add this:

    class ActionController::TestCase
      include Devise::TestHelpers
    end

    at the bottom of the test_helper.rb file. Not that that would ever happen to me…

  • Rails come with some awesome assertion methods for writing tests:

    assert_difference("User.count", +1) do
      create_a_user
    end
    

    That asserts that the count of user was incremented by one. The plus sign is not needed, that’s just an integer, I add it to make things clear. You can mix several of this expressions into one assert_difference:

    assert_difference(["User.count", "Profile.count"], +1) do
      create_a_user
    end
    

    That works as expected, it asserts that both users and profiles were incremented by one. The problem I have is that I often found myself doing this:

    assert_difference "User.count", +1 do
      assert_difference "Admin.count", 0 do
        assert_difference "Message.count", +3 do  # We send three welcome messages to each user, like Gmail.
          create_a_user
        end
      end
    end
    

    That looks ugly. Let’s try something different:

    assert_difference("User.count" => +1, "Admin.count" => 0, "Message.count" => +3) do
      create_a_user
    end
    

    Well, that looks nicer, and straightforward, so I implemented it (starting from Rails 3 assert_difference):

    def assert_difference(expressions, difference = 1, message = nil, &block)
      b = block.send(:binding)
      if !expressions.is_a? Hash
        exps = Array.wrap(expressions)
        expressions = {}
        exps.each { |e| expressions[e] = difference }
      end
    
      before = {}
      expressions.each {|exp, _| before[exp] = eval(exp, b)}
    
      yield
    
      expressions.each do |exp, diff|
        error = "#{exp.inspect} didn't change by #{diff}"
        error = "#{message}.\n#{error}" if message
        assert_equal(before[exp] + diff, eval(exp, b), error)
      end
    end
    

    Do you like it? If you do, let me know and I might turn this into a patch for Rails 3 (and then let them now, otherwise they’ll ignore it).

    Update: this is now a gem.

  • I always dislike setting up cron jobs for web applications. The whole application lives in /var/www but I manually create this single line somewhere else that I have to remember to setup if I switch servers, and turn off if needed, and maintain accordingly. Well, it so happens that it can be done much better with the tools we already have: the usual cron and capistrano (or any other similar release tool).

    In Restraq, a project I’m working on, I created a file named crontab in the config directory with these contents:

    MAILTO=pupeno@restraq.com
    */5 * * * * www-data cd /var/www/restraq/current && RAILS_ENV=production rake mail:get
    

    Cron can read from some predefine files but it also reads every file in /etc/cron.d/. That’s a very common Linux pattern, the files in the directory with the .d extension are read as if they were one big configuration file. Other programs do it too.

    Then it’s only a matter of installing it there on deployment. At first I’ve tried with a symlink but it seems cron ignores symlinks so I decided to copy it with this line on my deploy.rb file:

    run "#{try_sudo} cp #{current_path}/config/crontab /etc/cron.d/restraq"
    

    and that’s it. No more fragmentation. Whenever, wherever I install my web app, there goes the cron jobs.

  • In Ruby on Rails there’s a very easy way to create a select tag:

    form.select("period", [["Month", 1], ["Year", 12]])

    In my case I have the options in a hash, like:

    periods = {
      1 => "Month",
      12 => "Year"
    }

    but when I did this:

    form.select("period", periods)

    I was surprised to find out that the keys of the hash are used as the content of the options, and the values as the keys, producing this code:

    <select id="resource_period" name="resource[period]">
      <option value="Month">1</option>
      <option value="Year">12</option>
    </select>

    Definitely not what I wanted, so I wrote this method:

    class ApplicationController < ActionController::Base
      helper_method :hash_for_select
    
      def hash_for_select(hash, sort_by = :value)
        sort_by = sort_by == :value ? 0 : 1
        options = hash.map { |k, v| [v, k] }.sort { |x, y| x[sort_by]  y[sort_by] }
      end
    

    and now I can do

    form.select("period", hash_for_select(periods))

    if I want the options sorted by key value or

    form.select("period", hash_for_select(periods, :key))

    if I want them sorted by keys.

  • Update: there’s a new version of this post covering Devise 4.0.0: Show a devise log in or sign up forms in another page

    Devise create various forms, among them one for signing up and one for logging in of course. These are the forms as they are generated in Devise 1.0.8:

    <h2>Sign up</h2>
    
    <%= form_for(resource, as: resource_name, url: registration_path(resource_name)) do |f| %>
      <%= f.error_messages %>
    
      <div>
        <%= f.label :email %><br>
        <%= f.text_field :email %>
      </div>
    
      <div>
        <%= f.label :password %><br>
        <%= f.password_field :password %>
      </div>
    
      <div>
        <%= f.label :password_confirmation %><br>
        <%= f.password_field :password_confirmation %>
      </div>
    
      <div>
        <%= f.submit "Sign up" %>
      </div>
    <% end %>
    
    <%= render partial: "shared/devise_links" %>

    and

    <h2>Sign in</h2>
    
    <%= form_for(resource, as: resource_name, url: session_path(resource_name)) do |f| %>
      <div>
        <%= f.label :email %><br>
        <%= f.text_field :email %>
      </div>
    
      <div>
        <%= f.label :password %><br>
        <%= f.password_field :password %>
      </div>
    
      <% if devise_mapping.rememberable? %>
        <div>
          <%= f.check_box :remember_me %>
          <%= f.label :remember_me %>
        </div>
      <% end %>
    
      <div>
        <%= f.submit "Sign in" %>
      </div>
    <% end %>
    
    <%= render partial: "shared/devise_links" %>

    If you try to put them somewhere else you’ll run into some problem. There are some variables/methods those forms use that you’ll be lacking, specifically: resource_name, resource and for logging in also devise_mapping. I’ve recently tried to put them both in the homepage for an upcoming project of mine and this is how I’ve solved it:

    module ContentHelper
      def resource_name
        :user
      end
    
      def resource
        @resource ||= User.new
      end
    
      def devise_mapping
        @devise_mapping ||= Devise.mappings[:user]
      end
    end
  • When I start coding a Ruby on Rails project, I find myself modifying the migration files over and over. I know this is not the way they were intended to use, but to avoid upfront design, I only ad fields when I need them. If I respected the way migrations were intended I would end up with hundred of migrations the first day and I would waste half my day just creating migrations.

    After a project is deployed or a second developer is working on it, I revert to the way migrations are intended and I create a new one every time there’s a change I need in the database.

    As migrations are intended to run only once, if you modify them, they won’t get run; and if you force them to run them, they’ll fail, because the database already contains such a table. So I’ve found myself doing this quite often:

    rake db:drop && rake db:create && rake db:migrate && rake db:seed && rake db:data

    db:data is a task I created to generate some sample data. Good known data that I can use to test the site locally. I’m using Factory Girl to create it, which I also use for the tests so I can re-use as much data creating logic as possible. It’s very good to get to a known state of the project you are developing and to get other developers started right away. I really recommend everyone doing it.

    The problem is that I also need to reset my test data, so I end up having this other command and it gets horrible:

    RAILS_ENV=test rake db:drop && RAILS_ENV=test rake db:create && RAILS_ENV=test rake db:migrate && RAILS_ENV=test rake db:seed

    Note: no db:data this time.

    I’ve got tired of re-writing these commands or trying to find them in my bash history, so I decided to write a Ruby task that will do it for me and here it is in case you want to use it too:

    namespace :db do
      desc "Crush and burn the database"
      task :hard_reset => :environment do
        File.delete("db/schema.rb")
        Rake::Task["db:drop"].execute
        Rake::Task["db:create"].execute
        Rake::Task["db:migrate"].execute
        Rake::Task["db:seed"].execute
        if !Rails.env.test?
          Rake::Task["db:data"].execute
        end
      end
    
      desc "Generate sample data for developing"
      task :data => :environment do
        # Create the sample data in here
      end
    end

    Enjoy!

    Update: To delete all records without resetting the whole database, check my post Deleting all records in a Rails project.

  • I’m reading the book Refactoring and one of the refactorings it shows is called “Consolidate Duplicate Conditional Fragments” and it shows an example in Java:

    if (isSpecialDeal()) {
      total = price * 0.95;
      send();
    } else {
      total = price * 0.98;
      send();
    }

    is refactored into

    if (isSpecialDeal()) {
      total = price * 0.95;
    } else {
      total = price * 0.98;
    }
    send();

    If you do it in Python it’s actually quite similar:

    if isSpecialDeal():
      total = price * 0.95
      send()
    else:
      total = price * 0.98
      send()

    is refactored into

    if isSpecialDeal():
      total = price * 0.95
    else:
      total = price * 0.98
    send()

    But in Ruby it’s different. In Ruby, like in Lisp, everything is an expression, everything has a value (maybe there are exceptions, I haven’t found them). Let’s look at it in Ruby:

    if isSpecialDeal()
      total = price * 0.95
      send()
    else
      total = price * 0.98
      send()
    end

    is refactored into

    total = if isSpecialDeal()
      price * 0.95
    else
      price * 0.98
    end
    send()

    Or if you want it indented in another way:

    total = if isSpecialDeal()
                  price * 0.95
                else
                  price * 0.98
                end
    send()

    We can push it one step further:

    total = price * if isSpecialDeal()
                              0.95
                            else
                              0.98
                            end
    send()

    Of these three languages, only Ruby can manage to have inside each branch of the if only what changes depending on the condition and nothing else. In this simple case you could use the ternary operator, :?, but if the case wasn’t simple, Ruby would be at an advantage.

    I’m reading Refactoring: Ruby Edition next.

  • I strongly believe in revision control. Back in the days when there wasn’t any other choice that CVS I spent countless hours setting up CVS servers and re-freshing my mind on its obscure commands and arguments, which were almost all of them. If you drop my copy of Professional Linux Programming it’ll open on the CVS chapter by itself (much like my copy of The C Programming Language will open itself on the pointers chapter, somewhere on the pointers to pointers section).

    Then came Subversion and it was somewhat better. After that we got the explosion of revision control systems with the introduction of distributed ones. We’ve got Darcs, Git, Mercurial, Arch, Bazaar, etc. My choice right now is Git. It used to be Darcs, but unfortunately it stagnated for a long time and I moved on to the next new thing. I’ve used Mercurial as well. From my perspective Mercurial and Git are almost equivalent.

    For me distributed revision control was a breakthru. There’s one small feature that makes a huge difference for me. In the old days I used to set up CVS or Subversion servers/repositories (in those, a repository is essentially a server-side thing). I had to decide where that repository was going to reside, how it was going to be named, how it was going to be accessed (ssh, http, custom protocol?), by whom, etc.

    Today with Git I just do this

    git init

    That’s it. I’m done. The directory where I am is now a repository. I can start committing, branching, rolling back, looking at the history, generating patches, stashing, etc. It’s that simple. The fact that it’s so simple goes from a quantitative advantage to a qualitative advantage. What I mean is that I not only revision-control things faster but that I revision-control things I wouldn’t have otherwise. For example the configuration directories on my servers.

    I know many people will complain: “But with no centralized repository chaos will ensue”. The fact that it’s distributed doesn’t mean there can’t be a central repository. When I want to collaborate with someone using Git I create one repo somewhere, in my servers or GitHub and we both push and pull from there. The difference is that I commit a lot locally, and when the chances are ready I push them. That means that I can commit broken code, without worrying.

    At work we don’t use a distributed revision control system, we use a centralized one and we have a very string peer reviewing policy. My current tasks involve touching code in many different files in many different systems never getting familiar to any of them. That means that it’s common for my peer reviews to say things like “all this methods don’t belong here, these two should go there, that one should be broken into three different ones going here, there and somewhere else”.

    Now I have a problem. I can’t commit because my peers don’t consider my code ready. My code works but it has to be refactored in a very destructive way. What happens if during the refactoring it stops working. For example copying and pasting I loose a piece of code. I can’t roll back to my working state and start over. If we were using a distributed revision control system I could.

    So, being able to commit non-finished code locally while colaborating with other people is one of my other crucial features in DVCS.

    The third one is being able to branch locally. In a similar vein as the last example. When I find myself thinking about a very destructive refactoring that I’m not sure if it’s going to get me anywhere and worst than that is going to take me three days to do; I just create a local branch. I experiment in that branch. If at any time I get tired or I need to do anything else I go back to the main one. That is what branching is about.

    Why is locally branching better than globally or centralized branching? Well, one reason is that a local branch doesn’t have to make sense to anyone else. I don’t have to pick a name that’s descriptive for anyone else than me. I don’t have to justify myself for creating a branch with anyone else. Let’s suppose I had an argument with a co-worker where I believe something is doable and (s)he believes is not. Do I want him/her to see that I created a branch to prove him/her wrong? I don’t. And if I prove myself wrong I want to quietly delete that branch and never ever talk about it.

    But I am starting to go into the very hypothetical realm. In short, for me, DVCS is about this:

    git init

    and get to code.

  • I like Erlang and I like gen_servers, I end up coding a lot of stuff as gen_servers and I like behaviors in generally. I even wrote some myself. I haven’t used Erlang for a while. I’ve been playing with it in the last couple of days, hopefully I’ll announce something soon.

    I ended up creating my own gen_servers template to get started. It’s a very verbose one and I’m putting it here more for me to know where it is that anything else; but, maybe it’s of use for someone else:

    -module(module).
    -compile(export_all).
    
    -behaviour(gen_server).
    -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]).
    
    %% Public API
    
    start() ->
      gen_server:start({local, ?MODULE}, ?MODULE, [], []).
    
    stop(Module) ->
      gen_server:call(Module, stop).
    
    stop() ->
      stop(?MODULE).
    
    state(Module) ->
      gen_server:call(Module, state).
    
    state() ->
      state(?MODULE).
    
    %% Server implementation, a.k.a.: callbacks
    
    init([]) ->
      say("init", []),
      {ok, []}.
    
    
    handle_call(stop, _From, State) ->
      say("stopping by ~p, state was ~p.", [_From, State]),
      {stop, normal, stopped, State};
    
    handle_call(state, _From, State) ->
      say("~p is asking for the state.", [_From]),
      {reply, State, State};
    
    handle_call(_Request, _From, State) ->
      say("call ~p, ~p, ~p.", [_Request, _From, State]),
      {reply, ok, State}.
    
    
    handle_cast(_Msg, State) ->
      say("cast ~p, ~p.", [_Msg, State]),
      {noreply, State}.
    
    
    handle_info(_Info, State) ->
      say("info ~p, ~p.", [_Info, State]),
      {noreply, State}.
    
    
    terminate(_Reason, _State) ->
      say("terminate ~p, ~p", [_Reason, _State]),
      ok.
    
    
    code_change(_OldVsn, State, _Extra) ->
      say("code_change ~p, ~p, ~p", [_OldVsn, State, _Extra]),
      {ok, State}.
    
    %% Some helper methods.
    
    say(Format) ->
      say(Format, []).
    say(Format, Data) ->
      io:format("~p:~p: ~s~n", [?MODULE, self(), io_lib:format(Format, Data)]).
    

    Update 2010-01-04: I keep changing the actual contents of the template as I’m coding a little Erlang app. I’ve recently added, among other things, a call to get the state of the process, useful for debugging.