Using jQuery and Ajax to load sub-sections of a webpage

OK, so this isn’t really rocket science, but there are a couple of neat uses for this technique:

Ajax powered tab sets

Tabs can; broadly-speaking, be loaded in three different ways:

• Immediately loaded - i.e. when the webpage has loaded, it is the default selected tab.
• Later (deferred loading) - i.e. after the user clicks a tab, go off to the server and retrieve the contents of a tab, or
• Lazy loading - e.g. load in the background before the user clicks a tab.  This enhances the user experience as the contents of a tab are immediately shown, but on the flip side means requires more traffic and the user may never click the tab anyway, effectively wasting bandwidth.

Content panes, portal layouts etc

As with tabs, content panes on a page can be loaded in several ways, although lazy loading is probably not very useful.

Solution

Here’s a javascript (using jQuery) implementation of the above which supports loading now, loading on page load and lazy loading (which takes place after the on page loads):

ajaxLoader : {
    /** A queue of embeds to load on document ready */
    loadQueue: {},

    /** A queue of embeds to load on after the load queue */
    lazyQueue: {},

    /**
     * Called on Document Ready to load all the queued embeds
     */
    init: function() {
        jQuery.each(this.loadQueue, function(id, url) {
            wbHtmlWidgets.embed._load(id, url);
        });

        jQuery.each(this.lazyQueue, function(id, url) {
            wbHtmlWidgets.embed._load(id, url);
        });
    },

    _load: function(id, url) {
        $("#" + id).load(url);
    },

    /**
     * Loads a url into a dom id immediately - ensure that the page has
     * completely loaded before using this method, as manipulating the DOM
     * before it has been created does nasty things in browsers such as IE.
     * If you want to embed the wibl after the page has loaded, use embed.afterLoad()
     * @param {string} id The div id to load the embed into
     * @param {string} url The url to load into the specified div id
     */
    now: function(id, url) {
        this._load(id, url);
    },

    /**
     * Loads a url into a dom id after the onLoad queue has been processed.
     * @param {string} id The div id to load the embed into
     * @param {string} url The url to load into the specified div id
     */
    lazy: function(id, url) {
        this.lazyQueue[id] = url;
    },

    /**
     * Loads a url into a dom id after the page has loaded.
     * If you want to embed the wibl now, use embed.now()
     * @param {string} id The div id to load the embed into
     * @param {string} url The url to load into the specified div id
     */
    afterLoad: function(id, url) {
        if ( $("#"+id).html() == '' ) {
            $("#"+id).html('<image src="/images/spinner.gif" />');
        }
        this.loadQueue[id] = url;
    }
}

To get this to work, ensure that you have an onLoad (document ready) event on your page…

$(document).ready(function(){
    ajaxLoader.init();
});

Then to make the magic work, in your HTML you may want to do something like:

<span id="tab-0"></span>
<script type="text/javascript">
//<![CDATA[
    ajaxLoader.afterLoad("tab-0", "http://your/tab/address");
//]]>
</script>

Dumping and loading seed data for Rails database resets

As mentioned in our previous posts regarding database migrations in Rails, we have introduced a method of managing seed data in migrations. Although this methodology still has some problems, generally speaking it works pretty well.

After a database migration, a schema dump takes place. This allows fast recreation of databases without having to go through every single migration. The problem with this is that any seed data added during the migrations is not added back in when doing a db:reset. This can be solved by having a similar function to Schema Dumper.

Here are the seed data tasks (you probably want to add these into the db namespace):

namespace :seed_data do
  desc 'Load schema seed data'
  task :load => :environment do
    puts "== Loading dumped seed data"
    config = ActiveRecord::Base.configurations[RAILS_ENV || 'development']
    ActiveRecord::Base.establish_connection(config)
    require 'active_record/fixtures'
    Dir.glob(RAILS_ROOT + '/db/seed_data/global/*.yml').each do |file|
      Fixtures.create_fixtures(RAILS_ROOT + '/db/seed_data/global', File.basename(file, '.*'))
    end
  end

  desc 'Save schema seed datae'
  task :dump => :environment do
    config = ActiveRecord::Base.configurations[RAILS_ENV || 'development']
    ActiveRecord::Base.establish_connection(config)
    Database::SeedDataDumper.dump(ActiveRecord::Base.connection)
  end
end

You’ll need the SeedDataDumper class:

module Database
  class SeedDataDumper

    def self.dump(connection)
      new(connection).dump()
      true
    end

    def dump()
      @connection.tables.each do |table|
        if @connection.select_all("DESCRIBE #{table}").inject(false) { |b, col| b || (col["Field"]=="came_from_migration&quot  }
          data = @connection.select_all("SELECT * FROM #{table} WHERE came_from_migration IS NOT NULL&quot 
          if !data.empty?
            outfile = File.new("#{@folder}/#{table}.yml",  "w&quot 
            header(outfile)
            i = "0"
            outfile.write data.inject({}) { |hash, record|
              hash["#{table}_#{i.succ!}"] = record
              hash
            }.to_yaml
            footer(outfile)
          end
        end
      end
    end

    private
      def initialize(connection)
        @connection = connection
        @folder = "#{RAILS_ROOT}/db/seed_data"
        puts "== Dumping seed data"
      end

      def header(stream)
        stream.puts <<HEADER
# This file is auto-generated from all the current "came_from_migration" rows in the selected
# database.  It generates a yaml file per table which can be loaded in again using fixtures.
HEADER
      end

      def footer(stream)
        stream
      end

  end
end

Remember to put the following line at the end of your db:migrate task:

Rake::Task["db:seed_data:dump"].invoke if ActiveRecord::Base.schema_format == :ruby

Seeding data in Rails Migrations (Part 2)

We’ve had a significant number of hits on our first article on Seeding data as part of Rail database migrations which just goes to show that people are finding this a bit of a problem! As we have been using our technique for a few weeks now, we’d like to point out a few of the problems with it that we have found (some with solutions, and some that we just haven’t figured out yet!).

1. Fixtures#create_fixtures deletes data - Whoops! After looking at the source code, create_fixtures effectively drops all the data in the table it is just about to populate. Obviously this isn’t very useful if you want to use a migration on any table that isn’t empty.
2. Related data - Migrations in Rails 2 have a nifty feature allowing you to create inter-related table rows by using named data items. The methodology outlined in Part 1 doesn’t load all the yaml files into the migration and then run them, instead it runs them one at a time meaning (and with no model class) so Rails is unable to resolve the relationships between rows.

Solutions

The easiest way to fix the related data is to extend the reader for yaml files to have more than one table per file, consider a yaml seed data migration file “006_add_monkeys_and_fruit.yml”:

monkeys:
  george:
    name: George the Monkey
    pirate: reginald
    fruits: apple, orange, grape

fruits:
  apple:
    name: apple

  orange:
    name: orange

  grape:
    name: grape

This naming convention follows the ideas behind migration naming convention and doesn’t stop you having more than one seed file per migration (obviously of unrelated data) e.g. 006_add_tables_and_chairs.yml

Temporary work around

For the time being (until we extend the yaml parser) we have written a slightly different seed data function for our migration files This allows us to seed a row into the database, read its insertion ID and then reference that row object in subsequent inserts.

Conclusion

We still think that Fixtures provides a pretty good way of introducing structure data into our tables, but just needs a little bit more work before it plays nicely with migrations. We have also found the time to start writing the schema dump/load equivalents for seed data (when creating database from schemas, i.e. in production). These methods are just being finished and tested and we will post them up here in the next couple of days.

Simple client-side form validation in javascript using jQuery

I’m new to jQuery (I come from a prototype background), so I quite enjoyed my first little jaunt using jQuery… and I used it to create a simple client-side form validation library.

As we are generating a lot of our forms server side which in turn can be rendered into several different formats, so it’s important for us to have a simple way to attach validation to form elements.

For this method to work, each form element needs to be wrapped up in a container, a li is used in this example. Each container then has a class applied to it to indicate the type of validation required for the form field contained by the wrapper. Here’s the HTML snippet:

<ol>
	<li class="validation required"><label for="name">Contact name</label></li>
	<li class="validation date"><label for="dob">Contact DOB</label></li>
</ol>

Now all that needs to be done is a validation function bound to each input field, that’s easy because each input field is wrapped in a container with the “validation” class. The second class is the type of validation that is required for the input field. To keep it generic, you can just eval the class name as long as there is a function to match, heres the javascript:

init: function() {
	$('li.validation').find(":input").each(function() {
		var validations = $(this).parent().get(0).className.split(' ');
		for (var i=0; i<validations.length; i++) {
			vtype = validations[i];
			if (vtype != 'validation' ) {
				$(this).bind("blur", {element: this}, eval(vtype));
			}
		}
	})
},

required: function(o) {
	var element = o.data.element;
	// do some validation with element.value
},

date: function(o) {
	var element = o.data.element;
	// do some validation with element.value
}

The validation functions can then do whatever you want. I generate a boolean result and an error message (if necessary) and pass these parameters to a generic function which indicates the problem with the field, but equally as important may be to disable the submit button etc.

To stop any nasty conflicts, I’d recommend wrapping that all up in a namespace or similar. Other points worth nothing: this setup validates everything on a blur event (which might be a bit annoying). Also, there is no way of passing any parameters to the validation functions; so special cases need their own validation functions. However, I can’t imagine there will ever be that many different types of validation for it to matter too much.

Creating common fields in your Rails model database migrations

Sometimes you want a set of common set of fields in a bunch of your tables. DRY it up a little so you can write this in your migrations:

include MigrationHelpers

class CreateUserTales < ActiveRecord::Migration
  def self.up
    create_table :user_tales do |t|
      t.common_fields
      t.integer :tale_id, :null => false
    end
    add_common_field_indexes :user_tales
    add_index :user_tales, :user_id
    foreign_key :user_tales, :tale_id, :tales
  end
end

You need a /lib/migration_helpers.rb containing this:

module MigrationHelpers

  # Common fields for which we want to create an index by default.
  # Rather than create indexes for every field we create them by default only for those fields which do not change every time the record is updated.
  COMMON_INDEX_FIELDS = %w(owner_id created_by created_at)

  class ActiveRecord::ConnectionAdapters::TableDefinition
    def common_fields
      self.integer :owner_id, :created_by, :updated_by
      self.integer :version, :default => 0, :null => false
      self.boolean :is_deleted, :default => false, :null => false
      self.timestamps                   # adds created_at and updated_at as datetimes
    end
  end

  # Create indexes for the Common Fields
  def add_common_field_indexes(table_name, columns_to_index=COMMON_INDEX_FIELDS)
    columns_to_index.each do |column_name|
      index_name = "index_#{table_name}_on_#{column_name}"
      execute "CREATE INDEX #{index_name} ON #{quote_table_name(table_name)} (#{column_name})";
    end
  end

  # Hide mysql's foreign key syntax behind a simple method
  def foreign_key(from_table, from_column, to_table)
    constraint_name = "fk_#{from_table}_#{from_column}"
    execute %{ALTER TABLE #{from_table} ADD CONSTRAINT #{constraint_name} FOREIGN KEY (#{from_column}) REFERENCES #{to_table}(id)}
  end
end

Seeding data as part of Rails database migrations

So, we stumbled across a slight problem when trying to seed data as part of our database migration scripts. The de-facto standard seems to be to keep the seed data in yaml files and load them in using fixtures. This is a perfectly adequate way of doing it for seed data that will never change, but doesn’t really fit into the infrastructure of migrations. There are also many plugins out there to help with using fixtures to seed data, but none of them allow you to have different seed data for different migration versions.

After bumping our heads together, we came up with a way of managing seed data using the same philosophy as data structure migrations. The only prerequisite to your data structure is to add a column named “came_from_migration” to any table that is going to contain seeded data. This way when moving down migrations it is possible to determine what data that shouldn’t be there (and delete it).

1. Add a “fixtures” folder to /db/migrate. Your seed data yaml files will be saved here.
2. Create your seed data yaml file and name it in a similar fashion to your migration script using the table name as the descriptor e.g. 005_table_name_to_seed.yml. Save this in the folder you created above. This means that you can seed several tables as part of one migration version and it’s nice and easy to see what’s going on from just the file name.
3. Now I needed to write a couple of helper methods to move to and from the seed data. I’ve placed these in a MigrationsHelper module which is included in any migration that needs to seed data. And because we are so nice, here they are:

  def seed_from_yaml(table_name)
    info = get_migration_file_info(caller[0])
    fixture_folder = RAILS_ROOT + "/db/migrate/#{info[:type]}_fixtures"
    fixture_file = "#{info[:version]}_#{table_name}"
    puts "Seeding #{table_name} from #{info[:type]} fixture version #{info[:version]}"

    require 'active_record/fixtures'
    connection = ActiveRecord::Base.connection
    seed_data = Fixtures.new(connection, table_name.to_s, nil, File.join(fixture_folder, fixture_file))
    seed_data.insert_fixtures
  end

  def deseed(table_name)
    info = get_migration_file_info(caller[0])
    puts "Deseeding #{table_name} with anything newer than #{info[:version]}"

    execute %{DELETE FROM #{quote_table_name(table_name)} WHERE #{quote_column_name('came_from_migration')}=#{info[:version].to_i}}
  end

  private
    def get_migration_file_info(file)
      file.gsub!(/:.*/, ''  # get rid of everything after the colon
      migration = file.split '/'
      db_type = migration.include?("global&quot  ? "global" : "private"
      migration_file = migration[migration.length-1].split '_'
      migration_number = migration_file[0]
      { :type => db_type, :version => migration_number, :file => migration_file }
    end

Seeding data when not using migrations

What about when running tests or deploying, i.e. seeding data when not using migrations and coming from a empty database? Well, the schema should be held for you in your _schema.rb file, but that doesn’t cover any seed data. The answer is to create a *_seed.yml containing all the data in the database (in the same way the *_schema.rb file contains the complete data structure schema of the database). This way you can load in the seed data after the tables have been created. We have yet to write this method, so is left as an exercise for the reader at present. The basic premise is to go through every table in the schema, check for the existence of the “came_from_migration” column, do a “select * from table where came_from_migration is not null” on tables with that column and then concatenate the output from the select statements to a *_seed.yml and save in /db. There may also be problems with referential integrity with seed data and to what order it is inserted into the database. No doubt we will come across these difficulties soon.

Bibliography

• http://wiki.rubyonrails.org/rails/pages/FlexibleFixtures
• http://railspikes.com/2008/2/1/loading-seed-data
• http://quotedprintable.com/2007/11/16/seed-data-in-rails

Supported data types for create_table in Rails 2.0

Even given the sketchy documentation, I managed to glean the data types for create_table from the ActiveRecord::Migration Rails Framework Docs as being:

• :string
• :text
• :integer
• :float
• :decimal
• :datetime
• :timestamp
• :time
• :date
• :binary
• :boolean

A default value can be specified by passing an options hash like { :default => 11 }. Other options include :limit and :null e.g. { :limit => 50, :null => false }

Selenium RC environments with whitespace characters

We use selenium grid to orchestrate our acceptance testing over multiple browsers on multiple platforms. One of the nice things with registering environments with the selenium grid hub, is that you can use descriptive names like: “IE on Windows XP” and “Firefox on OS X” for example.

I was having difficulties with environment names with whitespace characters, so after a lot of experimentation, here are the tweaks I made to get it working across all Linux and Windows.

The Rakefile for the selenium-grid needs to have the rc_args method redefined as follows (notice the changes on lines 6 and 7):

def rc_args(options)
  args = []
  args << "-host" << (options[:host] || ENV['HOST'] || "localhost&quot 
  args << "-port" << options[:port]
  args << "-hubUrl" << (options[:hub_url] || ENV['HUB_URL'] || 'http://localhost:4444' 
  args << "-env" << "\"#{(options[:environment] || ENV['ENVIRONMENT'] || "*chrome")}\""
  args << "-env" << "\"#{(options[:environment] || ENV['ENVIRONMENT'] || "*chrome")}\""
  args << (options[:selenium_args] || ENV['SELENIUM_ARGS'] || "&quot 
  args
end

Now you can run the RCs from the command line as follows:

On windows:

rake rc:start PORT=5555 HUB_URL=http://172.16.7.122:4444 HOST=172.16.7.119 ENVIRONMENT=”IE on Windows Vista”


On linux:

rake rc:start PORT=5555 HUB_URL=http://172.16.7.122:4444 HOST=172.16.7.119 ENVIRONMENT=Safari\ on\ OS\ X

 

Welcome

Workbooks is developing a new application, delivered using the “Software as a Service” model. Eventually we’ll no doubt post more about what the application does, and who it’s aimed at - if you want to know more go to the Workbooks website. This is the blog of Workbooks’ development team; as we discover things which may be useful to the community in general we’ll publish them here.

Technically, we’re using Ruby on Rails and ExtJS as a starting point. We’re fans of the Apple MacBook Pro and use it to run VMware Fusion as a host for 64-bit Linux development systems.

There are a lot of blogs which are quickly created and become moribund. Then there are those which have a community of subscribers in the millions. While I doubt this will be the latter, we’ll do our best to fill it with useful technical information.