Posts

May 28, 2007
Friendly URL Performance In Rails

During a discussion with Nick we wondered what clean URLs cost in terms of performance. It’s certainly nice to use clean URLs such as /blog/my_post_title instead of numerical URLs such as /blog/123, for ease of remembering, and verbosity. So how to go about testing this?

Test Setup
- Macbook Pro 2.16Ghz C2D. 2GB RAM.
- Rails 1.2.3
- MySQL 5.0.37 (Query Cache turned off)
I started by creating a simple Rails model with only an id and a permalink of the style shown above, created by iterating over the OS X words file. Over a million instances of this were then created in the database. I then randomly selected 10,000 of these to be found by by my tests.

Non-Indexed = Slow!

By timing this using the Rails benchmarker script, and taking away the 6s of setup involved in each test, I got the following times from finding the 10,000 records:
- find(id) : 12.0s
- find_by_permalink(permalink) : 4453s (Over an hour)
I didn’t think these results were even worth generating a graph for, the difference is startling. Using the simple permalinks is almost 400 times slower than using the numerical IDs. But all is not lost.

Indexing = Fast

So how can we make this faster? The fastest way to do this is to add an index to the permalink field, you can do this in a Rails migration with the following:
```
  add_index :samples, :permalink
```
If you want an index on more than just one field, feed the second attribute an array of field names.

The graph below shows just how close using an index on the permalink gets us. 1.8s extra for our 10,000 finds, with nice URLs rather than numeric IDs. That’s pretty acceptable, especially for a production app that needs to be fast. But can we make it faster?

Configuration Over Convention

The rails find_by_foo methods are nice, but there are faster options. How about manually setting the conditions?
```
  find( :first, :conditions => [ ":permalink = ?", permalink ] )
```
Or how about doing all of the SQL manually?
```
  find_by_sql(["SELECT * FROM samples WHERE permalink = ?", permalink])
```
So what does this give us? By just setting the :conditions manually, we get down to 12.1s, scratching at the door of the ID based find. And the entirely manual SQL? 10.9s, a whole second faster than our original query. (Obviously we can manually rewrite the find query to use SQL, making it as fast, if not faster)

Conclusion

If you are going to use permalinks or search on some other field of your tables, make sure to use an index. Other speedups could be obtained by writing the SQL yourself, but these are too small to be worth the hassle if your queries become more complicated in the future.

P.S. Loving the WIFI on GNER Trains, posted at over 100mph!
May 24, 2007
Mocking User_Agent In Rails Tests

Sometimes you may want to check the user agent of a request in your Rails controller, be this for checking for spam, or only allowing services such as Feedburner to access your RSS feeds. Something like this:
```
def feed
  if request.user_agent !~"Feedburner"
    # redirect to feedburner
  else
    # display feed
  end
end
```
So what happens when you try to test this using the standard Rails functional tests? You should see an error something like the following:
```
ActionView::TemplateError: undefined method `user_agent' for
 #<ActionController::TestRequest:0xb69d9b88>
```
Not so good. The solution is to add the following module to your app. I chose to add it in the same file as the appropriate controller, after the end statement for the class:
```
module ActionController
  class TestRequest < AbstractRequest
    attr_accessor :user_agent
  end
end
```
Now your tests should run fine, and you can even set the user agent as you wish in your tests:
```
def test_access_allowed_to_feedburner
  @request.user_agent = 'Feedburner'
  get :feed
  assert_response :success
end
```
Hope this helps you out.
May 17, 2007
Penrith Merida

At the weekend I was down in the Lakes for the Penrith Merida with the EUCC guys and gals.

We stayed at Tim’s parent’s house on the saturday, and enjoyed some wonderful home cooking and hospitality.

The Merida itself was a tough one, 83km, and 2400m of climbing, some of which involved some hike-a-bike. Once I got over the start line (2.5mins, must be on the front line next time) and passed a few hundred people I started to settle in. On the first climb I ended up with a group of fairly fast guys who I was near for most of the race.

Towards the end I started to get cramp, annoyingly while sprinting downhill a few km from the finish, so that slowed me down a bit there. I finished in 4 hours and 38 minutes, 39 minutes behind winner Nick Craig, and somewhere in the top 30 (Someone else calculated top 10). The results they give out aren’t terribly accurate and don’t separate the different distances.

It is definitely the toughest race I have done in a while, I’ve had to spend all this week resting so I can recover in time for the Fort William SXC this weekend.

On a side note, I got a pair of Marzocchi Corsa SL WC forks last week, Out of the box they weren’t very smooth, and it wasn’t till I took the remote lockout off I realised they were never fully unlocked. Some lube in the lockout pulley the cable runs round soon sorted that out, and they now work perfectly.
May 7, 2007
Cycling Catchup

Not had a post for too long, so I’m going to do a quick catch up on what I’ve been doing cycling wise…

SXC 2007 Round 2 – Dalbeattie

The second round of the SXC series was at Dalbeattie, one of the 7 Stanes best known for it’s rocky singletrack and steep slabs. True to previous form, the race course avoided most of this, and instead used a large amount of steep natural singletrack. Super dry conditions meant that some corners were deep sand instead of the slippy mud of previous years.

In the Expert/Elite race I got off to a good start, though lost a few places on the first descent when I ended up behind the tape after a bit of mis-direction. I also had cramp for the first 2 laps, so had to ease off the pace a bit. Once the cramp disappeared I was flying again, really enjoying the course.

On the last lap I decided to give it everything, locking my forks out and stomping on the pedals up the main climb. I passed several people here, and caught Andy Wardman, who I passed on a later section of fireroad to put almost a minute into him before the finish.

I was 13th overall, which compared to the top fives I was getting last year is a little poor. No doubt the field is stronger this year, as my form is definitely better than last.

Ingliston Crit Series

For the second year in a row I was riding the support races at the Cycling Scotland Ingliston Criterium Series. A bit of a mixed bag of results here…
1. First round I got stuck behind two crashes, losing me decent position in the field. Low 20s finish.
2. Second round I lost a contact lens on the 3rd last lap, making things slightly difficult. Low 20s finish.
3. I missed the third round due to the SXC race the next day.
4. Round 4 – Sat in the front of the field for the first few laps, helping to chase down breaks. Came back to the front near the end and held my position to take 10th in the sprint.
A Century

On saturday Tom and I went on a big hilly loop round Innerliethen, Biggar and Lanark. It ended up being 100miles, not bad for a 6 hour day in the saddle. I’ve never done 100miles before, but as with the other long distances I have done, I find that after about 70miles my legs just keep going.
Apr 24, 2007
Command Line Upgrade To Ubuntu Feisty Fawn 7.04

This guide is for those who are upgrading from Ubuntu Edgy Eft 6.10 to Ubuntu Feisty Fawn 7.04. To do other upgrades the instructions are similar except for step #6 where the names are (obviously) different.
1. Update Apt:
```
sudo apt-get update
```
2. Update packages:
```
sudo apt-get dist-upgrade
```
3. Restart to get the latest kernel etc:
```
sudo reboot
```
4. Back everything important up now! If you haven’t done so already. ;o)
5. Backup your apt configuration:
```
sudo cp /etc/apt/sources.list /etc/apt/sources.list.edgy
```
6. Open your source list with your favourite editor, in my case nano:
```
sudo nano /etc/apt/sources.list
```
7. Comment out any references to install CDs at the top using a hash at the start of the line(s).
8. Change every occurrence of the word ‘edgy’ to ‘feisty’.
9. Save the file. In nano hit ctrl + x, then hit Y.
10. Update Apt again:
```
sudo apt-get update
```
11. Upgrade to the new distro:
```
sudo apt-get dist-upgrade
```
12. Restart:
```
sudo reboot
```
Finished!

This sequence worked perfectly for me. Milage may vary, so make sure you follow step 4 and backup everything. Hope this helps someone out.

Apr 19, 2007

Site Search Using Google In Ruby On Rails

Normally for searching in a Rails app when people ask about searching, I suggest using Ferret, the only downside to this being that it can only search model data, and not static content that may be marked up manually.

Enter Google. They index all of your content, no matter how it is generate. So why not use them for the search?

Prerequisites

Make sure you have the Hpricot gem installed –

gem update hpricot —source \

http://code.whytheluckystiff.net

Make sure you include both of them in your environment.rb as follows…


require 'hpricot'
require 'open-uri'

Controller

Generating the query string and getting the results is simple enough to do in a controller method, like so…


Search site using google

  def google

@query = params[:id]
@start = params[:start] if params[:start]
@start ||= “0”
Site url as well as any other conditions you’d like.
	I chose to ignore all of my tag cloud pages, pagination pages and date pages.

    site = ‘douglasfshearer.com -“tagged with” -“posts by date” -page’

uri=“http://www.google.com/search?q=#{URI.escape(‘site:’+site+’ ‘@query’&start=’+@page.to_s)}”
html_result = open(uri)
parsed = Hpricot(html_result)
parse out the number of results.

    @no_results = parsed.to_s[/<\/b> of about (\d*)<\/b> from/,1]

@results = (parsed/“div.g”).map do |ele|
{:title => ele.at(“a”).inner_text,
:link => ele.at(“a”)[‘href’],
Huge fat hack alert. Use gsub to get rid of the weird stuff around the bold statements.

        :description => (ele/(“font”..“font/br”)).to_s.gsub(/\221/,‘’).gsub(/\222/,’’)}

    end

View

A very simple view for this can be done as so…

<%- if @results -%>
<h3> <%= @start.to_i+1 -%> - <%= @start.to_i+10 -%> of about <%= @no_results -%> results.</h3>

<%- @results.each do |r| -%>
	<h4><%= link_to r[:title], r[:link] -%></h4>
	<p><%= r[:description] -%></p>
<%- end -%>

<%= link_to 'Prev', :start => @start.to_i - 10 if @start.to_i >= 10 -%> | 
<%= link_to 'Next', :start => @start.to_i + 10 if @start.to_i < @no_results.to_i - 10 -%>
<%- end -%>

There you go. It’s just proof of concept, is a bit dirty in places, and uses the uri :id for the query string, but it works and has a few niceties such as pagination. Go play, and report back on how you get on.

Thanks

Core code, hpricot and inspiration by _why.

Nicholas Wright for regular expressions and other random banter.

A Word Of Warning

Don’t hit google with identical queries too often, you may find your IP is blocked by Google for a short period of time. Mine was, so this is probably best not used for a large production app. I would think this probably breaches Google’s T&Cs too.

Did you like my Ruby on Rails related article? Then why not recommend me on Working with Rails.

Apr 12, 2007
2007 AT&T Williams Website Live

It’s live! This has been a while coming (for us devs at least) but finally it’s here! Check it out at www.attwilliams.com, or read on for some more technical jargon.

The backend of the site is Ruby on Rails, running a MySQL database. The press-only site is an xhtml affair powered by rails, whereas the public consumer site has a flash front end fed via RSS feeds from the backend.

Why not use Flash remoting? This was considered at the start, but on the launch of the first portion of the site, the servers had to deal with 16,000 hits an hour for the first 72hours, and flash remoting would criple even a reasonably large cluster. Using RSS lets us generate the files when new content is added, and cache them so they only need to be server by a web server. Pretty neat.

I’m pretty proud of this, it’s been hard work and involved some late nights, but the end result has been worth it. Well done to all the team! They’re all chugging beers, but I’ve got to race later on, so I’m sticking to tea for the moment.

I’ll have more to say technically in the coming weeks, and we have some other features that will be added in the near future, some very interesting from a rails aspect.

Credits
- Justin Fanning
- Radha Stirling
- Max Haggenmiller
- Eddie Bosticco
- Jason Griffiths
Rails:
- Shaun Laughey
- Michael Christenson II
- Myself
- Nicholas Wright (Blog post)
- Kyle Drake
- PJ Cabrera
- Steven Holloway
- Brad Bollenbach
Flash:
- Tim Cooper
- Matt Folkard
HTML:
- Ben Millar
Apr 4, 2007
SXC 2007 Round 1 - Laggan

Sunday just past saw the first round of the SXC series at Laggan. It used the same course as last year apart from a new boggy section that replaced a very fast section of singletrack, but cut out some fireroad. This boggy section was rideable apart from a middle section which was quicker to run. There was also a huge deep puddle on exit, which caught a few people out.

For the third year I was riding expert, and having just come back from Cyprus, I we feeling good. As good as I was feeling, the standard was definitely up this year. I came in 11th, 18 minutes down on the winner. 7 laps didn’t seem much harder than the 6 we did last year, and I came away feeling fairly pleased with my efforts, and having something higher to aim for next time.

I will get some photos up soon, but for the moment you can look at my (almost) complete 2007 race build.
Mar 28, 2007
Last Minute Tourism

As I was flying out late afternoon, I had time in the morning for a quick tour of Paphos. First of all I had to hit the shops to get a gift for Emma, don’t want her being disappointed. Luckily that didn’t take too long.

Once the pain was over, my aunt took me to the Paphos Archeological site, where we looked at the Roman (and earlier) mosaics at the House of Dionysos. These are some of the most awesome mosaics I have ever seen, the detail being amazing considering the materials that were being worked on.

It was interesting to see the animals, some of which seemed a little out of proportion, no surprises since they had no source materials such as photographs to work from, and the artist had probably only ever seen a dead board/tiger/leopard.

The second really interesting thing was the portrait of one particular gentleman (OK, I thought it was a woman with really big eyebrows at first). It was not so much the portrait itself by the small square columns surrounding it, created with perspective, something I believe the Romans were the first to depict in their art. You can see a photo of this on my flickr.

My flight back was with First Choice, and apart from the chaos in Paphos airport, it was pretty dull. Thanks to my dad and Babs for picking me up from Glasgow.
Mar 27, 2007
Last Day Of The Hills

Today I did a big loop to Polis and back again, in the opposite direction to that which I had done before. I also made a small detour to take some pictures of the concentration camp at Polemi. My unkle thinks this may be a relic of the 50s, now out of use. Why keep it though, and have a massive sign on the front of it?

The rest of the ride was fairly regular, apart from the last few days of riding starting to take it’s toll on me. I was getting twinges of pain in my right knee, no doubt due to the low (60-80rpm) cadence up the steep hills.

The bike is all packed up now, ready to go home. It’ll definitely need a wash when I get back.