Rails 3, Gems, Rake and Bundler

I ran into a bit of a mystery the other day. I have a new rails 3 (beta3) install.

I'm using Jeweler to build a new gem using Rake. When I install the gem using Rake all goes along as expected. If I go to the command line and type gem list. My gem does not show up. After digging around I found that the gem is getting installed into the ~/.bundler gem location.

If I run gem commands from the terminal then GEM_HOME is as I set it.
If I run gem commands from inside Rake (for my Rails 3 project that has had bundler install run on it) then it is using ~/.bundler/.. as the root of GEM_HOME.

As a result I have a gem that i need to install and re-install a lot. So I have to execute the gem uninstall commands from inside Rake.

Here is what my rake file looks like now:

Rails 3 templates and engines

What I really wanted to do was to setup a rails 3 environment that uses engines to organize modular functionality.

It has ALWAYS been my experience that any project will quickly get out of hand if the organization is not tackled very early. Rails projects get messy fast. If you are disciplined enough to use modules for functionality then you can avoid some of these issues. But I really don't like having module names that I have to use spread out in my code. (Albeit sometimes a module to separate functionality is the best course of action.) I've always wanted to have my own app namespace for my functionality boundaries. Wouldn't this be nice:

With plugins I got closer. With the gemifcation I got a little help packaging my plugins up and defining dependencies.

Then came along the rails engines plugin. I have to admit I liked it, but never felt comfortable diving in. The reason is that engine plugins required a fair amount of twidling bits in rails to get it working right. Whenever that happens I feel a little exposed. (Not that that is always a bad thing!) We'll good news for me...I waited long enough and rails 3 (and rails 2.3 as well) has engines as first class citizens.

I've spend a few days pushing them around and I like it. With a little work I was able to create some rails templates that will generate a new rails engine. I then configure the hosting app Gemfile to include my engine and I'm off and running. Nice!

I thought I'd share the work I've done for those that want to do something similar.

NOTE: All these samples are for postgres and rspec. On testing, there are many options out there. I happen to dig rspec, if you have not tried it you should. If you care about your data you will stop using mysql or sqllite and start using postgres.

You can find the templates @ http://github.com/voomify/voomify/templates.
(Warning they reference each other using a local path, so you will need to change that if you plan on using them for yourself. You can create a directory ~/dev/voomify/templates and put the templates there and they will work for you.)

If you have not checked out templates for rails 2.3 and rails 3 then you should do some reading. This is not a tutorial. Here are some references to get started with:
http://m.onkey.org/2008/12/4/rails-templates
http://asciicasts.com/episodes/148-app-templates-in-rails-2-3
http://benscofield.com/2009/09/application-templates-in-rails-3/

Here are some sample templates that really helped me get started:
http://github.com/jm/rails-templates

So what do these templates do? Well there are three templates:
1) app.rb - this creates a rails application with rspec and sets it up to be compatible with our engines.
2) engine.rb - this creates a rails application that is also an engine.
3) finailize.rb - this finalizes our application or engine. It is always called at the end of the process.

I then created a litle bash script that will make it easy for me to call these.

Normally I have to put together a long command line:


With this bash script (in ~/.bashrc on ubuntu):

Credit for this goes to: Ryan Bates (http://asciicasts.com/episodes/148-app-templates-in-rails-2-3)

My command line is now much simpler (whew!):


OK time to get cooking. First let's create a rails 3 engine. This engine will be embedded in our application container. It will be setup with rspec and all the proper '.gitignore' files. It will prompt you for the engine name and the database username. (It also adds jeweler to the app gems. We need this because our engines use jeweler to gemify themselves. Maybe in a later post I will figure out how to remove this dependency.)

Side Note: With rails 3 you no longer have to do that messy 'script/generate rspec_XXXX' junk anymore. Rspec is now installed as the default generator and you can simply run 'rails g scaffold ...'. I dig it.



Now let's add some job scaffolding:



The way these engines are setup they are also full blown applications that we can test inside. This way we develop the engine functionality just like we are in any other application. We can run generators, write tests the way we are accustom to. From your perspective it is just a rails app. How easy is that?

Now let's see it running like any other rails app:



Let's goto the jobs and see them in action: http://localhost:3000/jobs

Before we embed this we need to create a gem.


Ok let's embed this into our 'host' application.

Generate the host application:



Add the following to your Gemfile:



Ok now for migrations. We don't have any helpers for migrations yet. So for now you need to copy your engine migration into your app. Something like this:



Now startup your 'host' app and you will see your jobs. Nice! Imagine how modular you could be! Like a lego version of your coding self.



Let's goto the jobs and see them embedded inside our application as a rails 3 engine!: http://localhost:3000/jobs


Using the voomify-jobs.gemspec in the jobs engine directory you can specify gem depenencies. So if your engine depends on other gems you just need to add them here. That includes other engines. Now you can explicitly manage your dependencies. I feel so clean ... you?

Happy engines.

Note: I ran into a bundler, rake, gem install issue that I cover in this post:
Rails 3, Gems, Rake and Bundler

Setting Rails 3 Beta on Ubuntu 9.10

I spent some time trying to get rails 3 beta 2 running on Ubuntu.
The default gem install using apt-get is 1.3.5.
For rails 3 you will need gem 1.3.6.

Install RubyGems 1.3.6


OK now you are ready to follow the rails guide:
http://weblog.rubyonrails.org/2010/2/5/rails-3-0-beta-release/

Ok Now to install rspec:
If you don't have xmllib already setup you will get errors building nokogiri. Follow these to fix that:

http://nokogiri.org/tutorials/installing_nokogiri.html

Then install rspec-rails 2.0:
http://github.com/rspec/rspec-rails

Happy Rails 3 Riding!

Vendor rails and svn

We are using SVN and vendoring Rails. Everytime we update to a new version of rails it requires a bunch of deletes and adds. We do this for multiple projects. It is a pain. So I finally bit the bullet and decided to create a repository area for rails version. In our tree we put our rails versions off our trunk like the following:

/etorg/trunk/rails/rails-2.3.5

Then we us svn:externals with the path in the vendor directory.

The trick (and the reason I'm writing this down) is that to keep svn from either complaining with a warning or giving an error was to move the original vendor/rails directory to another name 'vendor/dead_rails' then commit. That results in no conflicts or warnings.

I hope this saves someone some head scratching time.

Erector - ruby CRUD views

In my last post I created a simple Erector view with a layout and talked about the pro's and con's of using a ruby class based view. Before we go any further I want to re-establish that I'm not a fan of markup. So if you are a markup jockey ... then you might want to move on. This Object Oriented view business is probably not your cup of tea. After all who would want clean, testable code in their views? Besides me that is.

OK you decided that you care enough about your views to read on. Stick with it, its worth the ride.

Let's get started by using the default scaffolding to generate a customer model. Run the following:

script/generate scaffold customer first_name:string, last_name:string, company:string, phone:string, email:string

Then run the migration and test the pages.

rake db:migrate

Now lets turn these views into erector views. Erector comes with a tool that will turn your erb's into erector classes. Run the following from your application root directory:

erector app/views/customers/**

If you look in your views/customers directory you will now see .rb files. Let's compare the code. Here is our show.html.erb:

<p>
  <b>First name:</b>
  <%=h @customer.first_name %>
</p>

<p>
  <b>Last name:</b>
  <%=h @customer.last_name %>
</p>

<p>
  <b>Company:</b>
  <%=h @customer.company %>
</p>

<p>
  <b>Phone:</b>
  <%=h @customer.phone %>
</p>

<p>
  <b>Email:</b>
  <%=h @customer.email %>
</p>


<%= link_to 'Edit', edit_customer_path(@customer) %> |
<%= link_to 'Back', customers_path %>

Now here is the erector equivalent:

class Views::Customers::Show < Erector::Widget
  def content
    p do
      b do
        text 'First name:'
      end
      text @customer.first_name
    end
    p do
      b do
        text 'Last name:'
      end
      text @customer.last_name
    end
    p do
      b do
        text 'Company:'
      end
      text @customer.company
    end
    p do
      b do
        text 'Phone:'
      end
      text @customer.phone
    end
    p do
      b do
        text 'Email:'
      end
      text @customer.email
    end
    rawtext link_to('Edit', edit_customer_path(@customer))
    text '|'
    rawtext link_to('Back', customers_path)
  end
end

It is pretty straight forward. One gotcha is that the base class is the
Erector::Widget class. The problem is that we are using rails helper and these are exposed using the Erector::RailsWidget base class. You can either change the base class to Erector::RailsWidget or you can introduce an erector layout that derives from Erector::RailsWidget and then you would derive from your layout class. (See my last post for more info on this.) The other thing you need to do is to move or destroy the erb files. If you don't move or delete them then your new erector view classes will not be found. Now you have erector views for your scaffolded customer. Take it for a spin.

I personally think the syntax is cleaner if we use the {} block syntax instead of do end. Here is the same view re-written to use {}:

class Views::Customers::Show < Erector::Widget
  def content
    p {
      b {text 'First name:'}
      text @customer.first_name
    }
    p {
      b {text 'Last name:'}
      text @customer.last_name
    }
    p {
      b {text 'Company:'}
      text @customer.company
    }
    p {
      b {text 'Phone:'}
      text @customer.phone
    }
    p {
      b {text 'Email:'}
      text @customer.email
    }
    rawtext link_to('Edit', edit_customer_path(@customer))
    text '|'
    rawtext link_to('Back', customers_path)
  end
end

Now you have a feel for what it looks like. How about a quick refactor of this code. I personally don't like the hard coded labels, so let's remove that:

class Views::Customers::Show < Erector::Widget
    
  def show_column(col)
    p {
      b {text col.to_s.titleize }
      text @customer[col]
    }
  end
  
  def content
    show_column :first_name       
    show_column :last_name
    show_column :company
    show_column :phone
    show_column :email
    
    rawtext link_to('Edit', edit_customer_path(@customer))
    text '|'
    rawtext link_to('Back', customers_path)
  end  
end

Now that's what I'm talking about. Now gee, if i do that in all my show views, I can easily refactor that method and make it available in either a base class or a module mix-in. What I like is how easy the refactoring is. It feels as it should easy.

Rawtext and text

You'll notice that we output using either text or rawtext. The difference is text is escaped and rawtext is not.

Erector - Object Oriented views

I just got back from railsconf. On the plane ride home I decided to take a look at Erector. Erector, besides having a cool name, is a dsl for writing markup. It is very similar to markaby, except erector views are not templates, they are plain old ruby objects. Let's look at an example:

class Views::Home::Index < Views::Layouts::Application
 def main_content
   h1 'Welcome to Voomify'
   p do
     text 'What is Voomify?'
   end
   p "Voomify is the verb for applying Voom to a problem."
  
   p "Voom as defined by Dr Seuss in 'The Cat in the Hat Comes Back':"
  
   blockquote do
     p "'Voom is so hard to get,
           You never saw anything
           Like it, I bet.
           Why, Voom cleans up anything
           Clean as can be!'"
    
     p "Then he yelled,
           'Take your hat off now,
           Little Cat Z!
           Take the Voom off your head!
           Make it clean up the snow!
           Hurry! You Little Cat!
           One! Two! Three! GO!'"
    
     p "Then the Voom...
           It went VOOM!
           And, oh boy! What a VOOM!"
    
    
     p "Now, don't ask me what Voom is.
           I never will know.
           But, boy! Let me tell you
           It DOES clean up snow!"
   end
   
 end
end

This will render the following code:

<h1>Welcome to Voomify</h1><p>What is Voomify?</p><p>Voomify is the verb for applying Voom to a problem.</p><p>Voom as defined by Dr Seuss in 'The Cat in the Hat Comes Back':</p><blockquote><p>'Voom is so hard to get,
           You never saw anything
           Like it, I bet.
           Why, Voom cleans up anything
           Clean as can be!'</p><p>Then he yelled,
           'Take your hat off now,
           Little Cat Z!
           Take the Voom off your head!
           Make it clean up the snow!
           Hurry! You Little Cat!
           One! Two! Three! GO!'</p><p>Then the Voom...
           It went VOOM!
           And, oh boy! What a VOOM!</p><p>Now, don't ask me what Voom is.
           I never will know.
           But, boy! Let me tell you
           It DOES clean up snow!</p></blockquote>

Admittedly this is not a very good example showing any benefit at all to using Erector. It is all markup and requires no real logic at all. Be patient we'll eventually get there.


Why would anyone want to do that? We'll for one I hate markup. (When I'm in full markup mode, i feel dirty.) If you love markup and erbs, then you should stop reading now. Before you leave, I'll leave you with this, you can easily mix erector and erbs (or any other template that works with rails). Some tasks are very well suited to markup and others are better suited to a markup dsl. If you are tag heavy then you should stick with your erb's. If you find you views are light with markup but heavy with ruby flow and control logic then erector or markaby may be the ticket.

Markaby or Erector

I dont' know much about Markaby. After taking Erector for a spin, I took a closer look at Markaby. I like it. The biggest difference to me is that Markaby allows you to create ruby template files the same way erb's do. The views are still templates with ruby syntax. They are not first class ruby objects. With erector they are first class plain old ruby objects. Why is this good? It gives you all the tools of inheritance and mixin's for your views. That is cool. Especially for an application with multiple views of the same underlying models. You can refactor your views into base classes that derive and render the same data in different ways. This is object oriented design for views. Nice.

Side Note

I've seen object oriented view code in other languages and it leads to some very powerful re-use that all OO programmers can understand. The most ambitious of these attemps was by an HR company named Seeker in the bay area. I was working for Concur at the time and we bought Seeker back in 1999. (It did not work out well, but that is a story for another time.) Seeker created their own markup language that was object oriented. The nature of HR data is that it has very complicated rules regarding who can see what data and when. The OO design of the language allowed that to be abstracted to the base classes and a functional programmer simply focused on the problem at hand. They took it further, as all commercial enterprise applications do, and they allowed the customer to define new models and views. Those views were very easy to write with this advanced data access logic abstracted out. Their customers loved it. They wrote very advanced business applications on top of this abstraction.

A Closer Look

So let's look a little deeper at using Erector with rails. Follow the erector installation and you'll be able to write an erector view. One gotcha I ran into was the fact that your views should derive from Erector::RailsWidget. If you don't do that you don't have access to the rails helpers.

To layout or not to layout

You have a couple of options for the layout. You can keep your erb layout and just drop in your Erector views. That is cool if you have an app already that you want to mix erector into quickly. It also allows the views to be rendered with different layouts.


The other option is to create an Erector layout. If you do that then you set your controler like so:

class HomeController < ApplicationController
  layout nil
...

Here is a simple erector layout:

class Views::Layouts::Application < Erector::RailsWidget
 def javascript_includes   
   javascript_include_tag 'application' 
 end

 def stylesheet_includes
   stylesheet_link_tag 'styles'   
 end

 def header
   h1 'Voomify'
 end

 # override this to render your view 'main' content
 def main_content   
 end
 
 def footer   
     a 'Home', :href=>"/"
 end

  
 def content
  
   html do
     head do
       title "Voomify"
       javascript_includes
       stylesheet_includes
     end
     body do     
      
       div :id=>"maincontainer" do                          

         div :id=>"contentwrapper" do
           div :id=>"topsection" do
             header
           end
         
           div :id=>"contentcolumn" do
               main_content
           end
           div :id=>"footer" do
             footer           
           end
         end                                                         
       end
     end
   end  
 end
end

Then you derive your view from the layout erector class. The view example above does just that. Notice that the view defined above implements main_content. This is the method defined by our layout, you can name it whatever you want. You'll also notice that it has methods to override for the header, footer, javascript_includes and stylesheet_includes. So if the page want's to modify any of those elements all it needs to do is override those methods. It can call super if it wants or just replace it.


This got me thinking about the decoupled relationship between templates and their view. The view does not know anything about where they are being rendered. Generally this is a very good thing. For example a view may be rendered on a page, or as an ajax call. Coupling the view to the layout creates a tight dependency between the two. This troubled me at first. (OK not that much, the world has bigger problems!) But then I started thinking about it. How often do I have single view that has a different layout? Not much. When I do with Erector I could use a decorator pattern from my base layout. OK that works. What about ajax forms? I have not tried it yet, but I'm pretty sure I can call my main_content to return to an ajax request.



OK so not very often do I have multiple layout variations with the same view. What does happen a lot is that I would like to make a modification to how the layout renders based on the view. Eventually the layout ends up with conditionals that render some of the erb if some variable is set by the controller. Yuck! How many times have you done that. (Be honest!) You say to yourself its ugly, but it's a template. You grit your teeth and move on. Or if it really bothers you, you introduce a helper, but it suffers from the same condition, branching logic, but now its written in ruby. An Object Oriented layout can easily eliminate that. Another case is when a view participates in another relationship. For example a given view may be related to other models in the system. The layout defines a standard layout for this relationship and then the view code implements main_content and related_content. The related_content method could return markup, or the layout may take care of the markup, and all the related content needs to do is return the model objects that are related to my current view.


In my next post I'll be building a simple contact model and then turning it into an Erector set of views. Until then ... keep it clean.

Railsconf 09 -Vegas

Here are my takes on the railsconf 09. It was my first railsconf. I've attended many JavaOne and MSFT PDC's, dating back to 1990. What struck me most was the enthusiasm of the community. You get that vibe in general, but to see a ballroom full of rails enthusiasts is another story.

In Java and MSFT events they are pushing new technology, even when it is barely baked. I expected this railsconf to do the same with rails 3, but there was not much content talking about rails 3. DHH did spend his keynote talking about some major highlights. My take is that rails 3 will be more elegant, flexible and performant, but the migration will be painful.

At the keynote DHH opened up talking about the attacks on rails. He was telling his on story about how he came to understand they were not personal attacks and it really just did not matter. It made me realize how early stage the rails community really is.(I'm not talking about the technology, but the community.) The vibe is: the rails world has been growing, but is it grown up enough to become a major player? You can especially hear it when people start talking about enterprise rails. Those conversations always start with a justification of how rails is ready.

The rails community at large is a passionate group that believes they have a better way of doing things. (Even if it is not entirely true.) It has a rebel feeling. I love that. The rails community needs to own its success. Strut around with more confidence. (BTW I'm NOT saying DHH does not have confidence!) Getting early adopters to use it is easy, getting wide spread adoption is much harder. Right now it still feels like early adoption. I'm not sure the rails community wants wide spread adoption. Careful for what you wish for.

Other highlights for me:
* Obie Fernandez being so passionate and honest about what he and hash rocket have gone through.
* Jim Weirich - Writing Modular Applications. Jim did a great job outlining a taxonomy for describing dependencies. The base material dates back to the 80's.
* Pen & Teller - They rock.