Happy Birthday issue 38204

Following on from my previous post there has been a flurry of activity on issue 38204 in Google Chrome’s bug tracking system.  However, I have since realised that issue 38204 originally advocated giving the end-user a setting to turn the status bubble on and off.  Although this is a useful feature by itself, it does not address the points of my post:

• usability: the status bubble covers up parts of web pages and web applications, impacting their usability.
• functionality: the status bubble moves out of the way when the mouse approaches it, which is just a workaround for the usability problem.
• accessibility: the status bubble appears, disappears and jumps around attempting to mitigate the usability problems, but in doing so causes distraction and confusion, especially for end-users with ADHD and ADD.
• security: the status bubble’s behaviour can be spoofed by a malicious website, potentially enabling a social-engineering exploit.

I believe that these aspects of the status bubble are much more serious than allowing the end-user to turn it on or off.  Therefore, and to celebrate the 1^st birthday of issue 38204 on March 15, I have given birth to another issue in Google Chrome’s bug tracking system: issue 75832

If you want to sing ‘Happy Birthday’, then be my guest, but please visit issue 75832 and show your support there too.

The status of recent browsers

The web is the application

The promise of the Web2.0 revolution was to enable websites to offer much greater interactivity to end-users – stepping significantly beyond the original hypertext design of HTML where each click results in a new page loaded into the browser.  It enabled applications to proliferate, spawning well-known services such as Gmail, Google Docs, Google Maps, Photoshop.com, Flickr, Vimeo, Facebook, etc.

At Workbooks we have gone much further – our service offers a full windowed desktop that looks nothing like a traditional web page.  There is a ‘Start’ button in the bottom left corner which opens a menu of ‘programs’.  Windows appear on the ‘desktop’, and can be moved around and resized.  Buttons, form fields and grids of data behave exactly as you would expect from an equivalent desktop application.  It is all very familiar to users of Microsoft Windows and other desktop operating systems.  If you’d like to try it, wander over to our demo login.

Once inside the desktop you forget that you are using a web application, and the same is true for many web applications.  Of course, all of this power is enabled by our faithful web browsers, and yet if you look at the screenshot above you can’t actually see the web browser.  In fact using Workbooks in full-screen mode is great – you get full use of the entire screen and the Workbooks desktop becomes your real desktop.

Powered by the browser

And for the past few years there has been a race amongst the web browser developers to improve their software in every way possible to support web applications.  Microsoft is about to release IE9, Google has got to version 9 of Chrome, Mozilla’s Firefox 4 is nearly ready, Opera is already at version 11.  Recently most of them have:

• added support for CSS3 and HTML5 to enable prettier and more capable web applications
• increased compatibilty with the myriad standards of the web to make developing web applications easier, e.g. Acid3 compliance
• added hardware acceleration of a variety of graphical tasks further increasing the speed of web applications by moving those tasks from the processor to the graphics card
• increased Javascript performance to stratospheric levels to enable web applications to run faster

In fact Google specifically wants to encourage web browsers to be very fast, interactive and responsive; their business model relies on providing services in the cloud.  As such, Chrome generally leads the pack in the race for performance, but also goes further than most of the others in making web applications easy to develop and to use.  Google defined Google Gears (a client-side database that could be used to develop off-line web applications), and then worked with the W3C to incorporate an equivalent feature into the HTML5 standard.  Google built support for desktop shortcuts into Chrome; the ability to save the URL for a web application as an icon on your real desktop so that when you double-click the application is loaded in full-screen mode – the browser gets out of the way.

The disappearing browser

It is clear.  Everything is moving to the web (or will be dragged).  Whether we like it or not, based on the faster communications, the smaller hardware and a myriad of enhancements in the browsers, the combination is enabling a transformation of the way we use applications. But while all of this is enabled by the web browsers, they are being increasingly side-lined as users want to access their web applications rather than the browser itself.

As I noted above, when using an interactive web application the user begins to ignore the browser – the normal web navigation tools such as the address bar, forward and back buttons, bookmarks, toolbars, etc. become irrelevent.

Google’s Chrome in particular made a splash when it appeared because it threw away a lot of the ‘chrome’ (user interface elements) that previously surrounded the web page.  They ditched the menu bar; got rid of the proliferation of toolbars that plagued Internet Explorer; combined search and addressing in a single field; combined buttons that can never be used at the same time (e.g. stop and refresh); pushed the tabs up into the window title bar.  Some users say they like this simplified user interface, but I wonder whether what they really mean is “I didn’t like so much of the browser getting in the way”.

Google has a site where they continue to develop their ideas on User Experience and Chrome is being influenced by Chrome OS and vice-versa.

All of this is good for web applications – every step is making them faster, more accessible, more interactive, more useable.

But…

Sorry to bring this lovefest to a screeching halt, but there is a problem.

One of the controversial user interface changes that Chrome introduced was that they threw away the status bar.  Until Chrome arrived every web browser had a status bar at the bottom of the window.  It served multiple purposes including showing what is currently being loading, when loading has finished and the URL that would be loaded if you click where the mouse is.  (It was also used by some web pages to display annoying messages, but let’s not go there!)  The status bar was a waste of space most of the time, so Google understandably wanted to get rid of it.  The problem is, where do we put its functions?

Many web developers liked the status bar showing each URL as it loaded – it confirmed that everything was working correctly.  Most users though couldn’t care less, and as a result many browsers replaced it with a progress bar, originally in the status bar, and later (I think it was Apple’s Safari) overlaid it in the address bar.  These days the servers, networks and browsers are so fast that even the progress bar is becoming redundant unless something is broken.  The spinner on the tab seems to be the future, providing feedback that something is happening, but giving no indication of when – it’s either instantaneous or it will be a long time!

Which really just leaves the function of showing where you will go if you click where the mouse is now (we’ll call this the ‘hover URL’ for want of a better term).  Chrome decided to add a little popup bubble inside the web page area, overlaying the web page.  As you can see on the left, this works quite well in a simple web page where there are few hyperlinks.

But there are just so many things wrong with it too:

• Impacting disabled users: The status bubble appears and disappears as the mouse moves over the links, which can be very distracting for some people.  Fading the bubble in and out helps a little, but if you have ADD (Attention Deficit Disorder) then this sort of user interface can become unusable.  Web applications now have an element appearing and disappearing over the application, and web applications tend to have loads of hyperlinks: ADD-sufferer’s hell.
• Supporting malware: Any web developer will tell you how easy it is to create a web page that has things that look like links to the user, but don’t behave like links.  They will also tell you how they can create a fake status bubble that appears when the user mouses over the fake link.  So now, do we really believe that screenshot above?  If I click on ‘About Google’, will it really take me to ‘about.html’ on Google’s site?  What if this were a site that looked uncannily like your bank’s website?  Malware heaven!
• Affecting the User Experience: Initially the status bubble got in the way, not allowing users to click on the elements of the web page underneath.  In a normal web page this is just annoying, but you can often simply scroll to work around it.  However, the Chrome developers considered this problem important enough that they implemented an enhancement to move the status bubble from the bottom-left corner to the bottom-right corner if the mouse went near it.
• Limiting web applications: Some web applications put all of their major controls at the top of the screen, e.g. Google Docs, whilst others use all parts of the screen, e.g. Workbooks.  As a result the status bubble can cover important parts of the user interface, no matter where the browser developers choose to put it.  And you can’t just scroll around it when the application is providing a desktop:
  

A bug was raised in Chrome’s bug-tracking database nearly 12 months ago (Issue 38204) and there have been some interesting suggestions for other designs that could solve the problem, but no movement in the development.

And the story would end there if I had written this a few months ago: Chrome hanging out there on it’s own, with a gently increasing market share, but not the market leader (yet).

Then Microsoft waded in with their size 9’s: they’ve copied the status bubble from Chrome into Internet Explorer 9.  Now this is serious.  Internet Explorer’s market share is steadly decreasing, but within 6-12 months we’ll probably have 20-30% of the web on IE9.

Firefox nearly fixed it

Mozilla’s early betas of Firefox 4 showed promise: they got rid of the status bar, but put its features in the address bar.  I’m sure that the first betas simply displayed either the hover URL or the current address in the address bar, but this screenshot from ghacks.net shows the later style of putting the hover URL on the right and truncating both URLs to fit in the available space.  Clearly this does not work when space is limited.

…which lead to Mozilla releasing Firefox 4 beta 12 last week and in some quarters amidst great fanfare that they had implemented the status bubble like Chrome!

…and yes they went through the same sequence of bugs that Chrome did until the Firefox status bubble also moves out of the way when the mouse approaches, etc.  They still haven’t completely solved a problem with the status bubble hiding the highlighted word when you search in the page.

Cue a Firefox bug in Bugzilla: Bug 637017 to match the Chrome bug.

Can we solve this?

This isn’t a technical problem.  There’s no complicated algorithm to design and code here.  It is simply a user experience design issue.  The requirements are:

• We need a way to display to the user the URL that they will access if they click on a link.
• It must be displayed outside of the web page area to avoid a host of other problems including accessiblity, enabling malware, usability, etc.
• It must not take up too much screen real-estate – all of the browsers are trying to reduce their pixels; that’s how we got here.

To me (and it is my somewhat uninformed opinion), there is a simple solution: when the user places their mouse over a link, display the hover URL in the address bar in a different colour, probably muted, e.g. grey, replacing the current page’s address.  This has several advantages: it takes no space that wasn’t already taken; it uses a space that is URL-sized and when the user does click, then the progress bar can run over the URL (à la Safari) converting it to a solid black URL as it is loaded.  It seems to me that this is an ideal combination of form and function that clearly informs the user what is happening.

Other solutions have been suggested in Chrome’s Issue 38204 and in Firefox’s Bug 637017.  Unfortunately I don’t have any sway with Microsoft so I can’t raise a bug there.

We need your help

If you:

• have other ideas of how to redesign the hover URL functionality, please add them to Chrome’s Issue 38204 and Firefox’s Bug 637017.
• have a preference for an existing design suggestion, then please add your opinion to Chrome’s Issue 38204 and Firefox’s Bug 637017.
• are a Chrome or Firefox developer, please consider raising the priority of this issue, and ideally getting it fixed.
• are a Microsoft Internet Explorer developer, please consider raising a bug and fixing it.

Warp factor 9

Workbooks is accessible worldwide, with any modern browser.  We have users in Europe, the USA and as far afield as the South Africa, Thailand and Mauritius.  The Desktop is a large amount of JavaScript, together with many assets.  So why is it still fast to load the Desktop?

• We’ve gone for extremely high-capacity network connections – recently upgraded by our provider to 10 Gbit/s – with low latency to our nearest major peering points and especially through LINX.
• All our assets are packaged and do not cache-expire – built using Scott Becker’s excellent Asset Packager plugin. So once they are downloaded they are not re-fetched until we release a new version of our software.
• Most interestingly, and most recently, we’ve adjusted our TCP/IP parameters to follow some neat tricks Google and Microsoft have been adopting recently…. I’d like to explain that here.

TCP/IP connections on a standard network stack often stall over high-latency (often this means “long distance”) network connections because the sending client waits for an acknowledgement packet after a small number of packets have been sent. Worse, this number doesn’t increase much and for very long: it reduces even on TCP connections which use HTTP keepalive.

The initial TCP window size has been bumped right up to 10 packets, which is a full 15K bytes (standard “slow start” settings would allow only 1 or 2 packets to be in transit before an acknowledgement is received).  So we can have up to 120K bits in flight before we need an acknowledgement packet back from the far end. This is almost always long enough for all the data in a server response to a client request so windows get drawn on the Desktop with the minimum of delay.

Finally we’ve now adopted a congestion-control technique called ‘CUBIC‘ which is much better for high-bandwidth but long-distance connections than our previously-adopted ‘Reno‘.

Background reading:

• http://news.ycombinator.com/item?id=1143692
• http://blog.benstrong.com/2010/11/google-and-microsoft-cheat-on-slow.html

Making an accessible API

We’ve been consumers of APIs from many vendors and projects, and so we have opinions on how things ought to work in an API. An API should be:

• powerful and comprehensive
• concise and easy to learn
• documented
• usable with the minimum of external dependencies
• stable

Then there’s the difference between a “wire protocol” to a web service and how you interact with a web service from within a programming language. Each programming language comes along with its own “style” – Ruby is heavily OO, PHP is a bit more workmanlike, VB is … well, VB.

Today we’ve published a set of PHP bindings for the API on github. You can now CRUD (“Create, Read, Update and Delete”) your objects in efficient batches in Workbooks from PHP and, better yet, there is accessible documentation and it’s released under the MIT license.

Please, go forth and innovate…..!

before(:all) doesn’t do what you’d expect

Like many before me, last week I was bitten by RSpec’s interesting implementation of before(:all). Although the documentation (http://rspec.info/documentation/) clearly states that before(:all) ‘is run once and only once, before all of the examples and before any before(:each) blocks’, it in fact runs once for every context in the scope. This might be useful, but in every situation I’ve seen it isn’t. The documentation goes on to say

Warning: The use of before(:all) and after(:all) is generally discouraged because it introduces dependencies between the Examples. Still, it might prove useful for very expensive operations if you know what you are doing.

which implies that it really does only get invoked once – you wouldn’t want an expensive operation to be executed for every context.

RSpec’s author has explained that the behaviour is actually as he intended it and proposed to change the documentation, but so far that hasn’t happened.

So, why did I need before(:all) to operate as advertised? I have written a test that collects some metadata and then iterates over it to generate common Examples for every attribute in the metadata. This is great because as the metadata is expanded, the test suite expands automatically ensuring consistent implementation of all of my API. The metadata is nested, so it makes sense to take advantage of RSpec’s contexts to nest the Examples so that the generated test results are self-documenting. Unfortunately, the setup for this test involves one expensive operation: login to the API to get the metadata and prepare to probe every attribute. Logging in takes several seconds. It should be faster, but that’s a different story – my test expands to nearly 5000 examples so even if logging in only took half a second we would still waste over 40 minutes. As it is, with RSpec running my expensive operation for every context, the test was taking over 4 hours!

A blog post helped me on my way, but it focussed on resetting the database between tests and didn’t work with Rails RSpec. I modified it to work with Rails RSpec and refactored it to make it useable in any test. Using it is simple; add the following to your spec_helper.rb file:

# HORRIBLE HACK to work around RSpec's broken before(:all) which does not get run
# once as per the documentation - it runs once at the level defined and once per
# sub-context!
# Extend ActiveSupport::TestCase with a 'before_once' that takes a block just like
# before(:all), but only gets called once at the top-level.
# Inspired by http://sickill.net/blog/2009/11/23/quick-and-dirty-hack-for-rspec-before-all.html
# but recast so that it can be used in any test.
class ActiveSupport::TestCase
  # Like before(:all), but only runs once at the top context and not for every
  # sub-context. It keeps track of the classes that are registered and does not
  # invoke the block in subclasses.
  def self.before_once
    _before_once_class_parents = []
    self.instance_variable_set(:"@_before_once_class_parents", _before_once_class_parents)
    before(:all) do
      unless _before_once_class_parents.select{|g| self.class.to_s =~ /^#{g}/}.any?
        _before_once_class_parents << self.class.to_s
        yield
      end
    end
  end
end

It adds a before_once method to ActiveSupport::TestCase that you can use just like before(:all), but it guarantees that it will only run once per scope.

So now, using before_once, I can ensure that my test only logs into the API once and not once per attribute. The net effect is a test that runs in about 90 seconds rather than taking half a day and climbing!

Ellipsis has an HTML tag…

From the “I never knew that…” department.

There’s an HTML tag for the ellipsis character.  Rather than writing “…” as above if you write … you’ll get a better-behaved …

It’s in HTML4.0 so it should be everywhere by now!

This is probably worth reviewing for similar goodies.

A milestone

We’ve just passed revision 10000 in our revision control system (Subversion) which is an opportunity for cake.   Which makes me wonder: what’s the largest revision number out there?

That sort of question implies something about revision control system you’re using …. with the advent of distributed systems such as Git perhaps this sort of question will one day be quaint.

But it’s interesting to note that the subversion project is now officially Apache Subversion so it is managed under the Apache Software Foundation’s Subversion repository and their trunk is currently at revision 982900!

Unexpected logout, expired cookies, sessions and the CKeditor

The Workbooks Desktop, different though it is to most web applications, is typical in that it uses cookies to track sessions. So when you log in a session cookie is stored in the user’s browser and from then on each request from the Desktop has the cookie included in the request.

But, starting a couple of weeks ago we started getting strange behaviour: some users reported getting logged out at various, seemingly random, times. We ruled out all the obvious causes, went through all the changes in our most recent release, but could not see what was going on. It wasn’t specific to any particular user, browser, request, network, operating system or time of day. Or anything else we could think of. We added diagnostic code and all that told us was that after a few hundred perfectly normal Ajax requests suddenly and without any apparent cause there was no cookie included with a request and our Desktop would respond by asking the user to login again. After a few logouts this rapidly becomes unfunny.

Eventually we discovered that we had literally dozens of cookies stored against our domain name in the affected browsers with names like scayt_1__options. A quick search through our source found that our new version of CKeditor had changed so that now the ‘SCAYT’ (Spell Check As You Type) plugin is enabled by default – and that integration creates literally dozens of cookies to hold your spelling preferences!

The browsers didn’t send all these cookies to us every time – they have different Paths to most of our app – but those browsers did discard the much more important Session cookie randomly – because browsers guarantee surprisingly little with regards to the retention of cookies.

Once we’d identified the problem the solution was easy: disable the plugin.

Strange problems with ruby-debug (rdebug) after installation of Ruby Enterprise (REE)

Some of us had some troubles with ruby-debug after upgrading to Ruby Enterprise Edition.  One of the issues was with ‘debugger’ statements within the source code causing the debugger to break, but to report line numbers multiplied by two!

To ensure you have a clean installation of ruby-debug that works alongside your shiny REE installation, follow these steps:

Remove all of the ruby-debug gems (including all versions if there’s more than one, and all executables):

sudo gem uninstall ruby-debug

Remove all of the ruby-debug-base gems (including all versions if there’s more than one):

sudo gem uninstall ruby-debug-base

Check there are no other rdebug executables lying around:

whereis rdebug

Remove any that you find

sudo rm /usr/bin/rdebug

Reinstall ruby-debug using the Ruby Enterpise Edition gem installer:

sudo /usr/local/bin/gem install rub

Essential TextMate Plugins

[tweetmeme service=bit.ly]
Our development team has now got upwards of 20 man-years experience with TextMate so we thought it would be a good idea to share with you the most useful bundles we’ve come across so far. If you feel there are any glaring omissions, then please leave a comment!

AckMate
http://github.com/protocool/AckMate

If you have worked with TextMate for any period of time, you will come to realise that the Find in Project is woefully inadequate. Fortunately, the plugin architecture means that a much superior alternative is available. Most programmers will have come across Ack, but for those of you who haven’t, it’s a technology that harnesses the power of Perls’ regular expressions and is very good at working with large trees of heterogeneous source code.

Once installed, the selected folder in the project drawer will be the root of the search and Ctrl + Alt + Cmd + F will open up the AckMate window. Results are displayed with a line above and below the matching text giving some context to the results.

See http://betterthangrep.com for more information on Ack.

RubyAMP
http://www.ruby-lang.org

RubyAmp adds a few productivity aids to TextMate. The most useful I find being the “Complete across tabs” (Ctrl + ; ). This works in a similar way to how pressing Escape will autocomplete within a file and provides the same functionality across multiple open files showing a popup list of possible matches, with the first 10 mapped to the number keys. Another really useful feature common amongst IDEs for statically typed languages is the ability to easily jump to method definitions, classes etc by using shortcuts.

ProjectPlus
http://ciaranwal.sh/2008/08/05/textmate-plug-in-projectplus

This bundle adds a few bells and whistles to the project drawer in TextMate. Instead of the standard fly-out drawer, ProjectPlus gives you a pane which remembers the state of your project tree and open tabs between opening and closing projects. One key feature adds SCM badges to the files and folders in the project drawer giving a quick and clear indication of files and folders within your SCM. Other small features include: “Show folders at top” instead of the default alphabetical order, “Sort by filetype”, Finder colour labels and QuickLook.

Installing Bundles

To install a .tmbundle file, place it in ~/Application Support/TextMate/Bundles and then select “Bundles > Reload Bundles” from TextMate. If you use TextMate with multiple OS X user accounts then .tmbundle files can copied to /Application Support instead.

Final note

On the subject of TM additions, by far my favourite syntax colouring scheme is the one used by railscasts.com available from here. Also, if you use code within presentations, it’s often really useful to copy blocks of code whilst retaining their style when pasting into another application. This is easily achieved using “Copy with style” available from here.