Douglas F Shearer

Rails Plugin: Acts_As_Indexed


Version 0.4.4 released 04 February 2008 – Fixed some minor AR bugs

This plugin allows ranked boolean-queried fulltext search to be added to any Rails app with no dependencies and minimal setup.

Install

./script/plugin install http://svn.douglasfshearer.com/rails/plugins/acts_as_indexed

Setup

Add acts_as_indexed to the top of any models you want to index, along with a list of the fields you wish to be indexed.


class Post < ActiveRecord::Base
  acts_as_indexed :fields => [:title, :body]

   ...
end

Searching

To search, call the find_with_index method on your model to search using the index. The optional ids_only parameter, when set to true, will return only the IDs of any matching records.


    # Returns array of Post objects.
    my_search_results = Post.find_with_index('my search query') # =>  [#<Post:0x314b09c @attributes={"...

    # Pass any of the ActiveRecord find options to the search.
    my_search_results = Post.find_with_index('my search query',{:limit => 10}) # return the first 10 matches.

    # Returns array of IDs.
    my_search_results = Post.find_with_index('my search query',{},{:ids_only => true}) # =>  [12,19,33...

Boolean Query Options

The following query operators are supported:

  • AND – This is the default option. ‘cat dog’ will find records matching ‘cat’ AND ‘dog’.
  • NOT – ‘cat -dog’ will find records matching ‘cat’ AND NOT ‘dog’
  • INCLUDE – ‘cat +me’ will find records matching ‘cat’ and ‘me’, even if ‘me’ is smaller than the min_word_size.
  • ”” – Quoted terms are matched as phrases. ’”cat dog”’ will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator. ‘cat -“big dog”’ etc.

Other Stuff

Need Pagination?

Then check out the matching will_paginate_search plugin.

Full Documentation

You can either build the rdoc by running rake rdoc in the acts_as_indexed directory, or look at the latest version online.

Problems, Comments, Suggestions?

All of the above are most welcome. dougal.s@gmail.com

Credits

Douglas F Shearer

Donate

If you find this plugin useful, please consider a donation (Paypal) to show your support!

Tags

, , , , , .

Related Posts

August 30th 2007 15:00 | comments (36)
 

Comments


Gravatar

Jim

September 12th 2007 22:29

Very promising plugin.

Are you able to index cross-models?

For example, if my "Post" model belonged to a "Category" model, could I index on the category_name field?

Gravatar

Jim

September 12th 2007 23:44

I just realized that I can do what I was asking about via an instance method.

So in the above example, the method would be:

******************
Class Post < ActiveRecord::Base

acts_as_indexed :fields => [:name, :post_category_name]

belongs_to :category

def post_category_name
category.name
end

end
******************

Very cool.

Next question. Is there a way to reset the index other than deleting the "index" folder?

Gravatar

Douglas F Shearer

September 13th 2007 09:41

Hi Jim.

Glad you figured out your own query, having instance methods makes a lot of things easy.

ATM there is no way to reset the index other than to delete it. Obviously filesystem access is not always available, so the next version will have a method to do this from within an application (If you look carefully in the source, there has been a stub for the method since the first version, just haven't got round to doing it).

Gravatar

Jim

September 13th 2007 16:56

Hi Douglas,

Thanks for the response. I have some more feedback after playing with the plugin some more this morning. If you'd rather chat through email versus blogs comments, let me know.

Feedback #1:
NOT search terms are modified when find_with_index is called. If I want to make two consecutive find_with_index calls on two different models, the second call is missing the NOT term.

Example of what I was trying to do below. The params SHOULD be the same in both cases.

**********************
#params[:q] => "cooking -japanese"
l = Lesson.find_with_index params[:q]

#params[:q] => "cooking "
s = Suggestion.find_with_index params[:q]
**********************

Feedback #2:
If my search has two terms and one of the terms is junk data, all of the results with the first term are returned. I would think nothing would be returned in this case.

For example, if I searched for 'dog asdfasdfas' (not quoted) all records with dog would be returned even though 'asdfasdfas' does not exist in the database.

Please feel free to email me if you have any questions. [jim at praexis dot com]

Gravatar

Douglas F Shearer

September 14th 2007 10:01

Hi again Jim.

I'll drop you an email, but as a brief response..

#1 Modified query param - That's a bug, will fix that just now.

#2 Default action for a space character is AND rather than OR. I can see what you mean about having it return 'dog' if 'asdfasdfas' doesn't exist at all, but most people would consider that to be unexpected behavior.

Gravatar

Raul Galindo

October 5th 2007 12:07

Hi Jim
I cant reach the address the plugin is in
is this because the server is down or am i doing something wrong?
i wait for ur reply
Best regards.

Gravatar

Douglas F Shearer

October 5th 2007 12:25

Hi Raul.

The svn server is working fine for me. No reason it should be down as it's hosted on the same machine as this site.

The plugin install command above should work in most cases, but if for some reason your permissions are wrong you may need to run it as..

ruby script/plugin

rather than

./script/plugin

Hope this helps.

Gravatar

cavatina

October 7th 2007 21:35

hi

svn server is not working

Gravatar

Douglas F Shearer

October 8th 2007 11:32

Working fine for me.

The svn address above WILL NOT work in a browser.

I've realised this is a problem for windows users without svn, so I may move to Apache Web-DAV or provide an archived download.

Gravatar

Jim

October 16th 2007 16:03

Hi Douglas,

I updated to your latest code yesterday. Nice improvements on the quoted searches and the relevance rankings. Nice work!

With that being said, I'm seeing a couple of issues when running tests.

Error #1: Errno::EDOM: Numerical argument out of domain - log
- Deleting the indexes and rerunning th e tests seems to fix the issue, but it has come back a few times.

Error #2: A file name "size" does not exist the first time I run the test with my existing indexes. Deleting the indexes and rerunning also fixes this error, but it seems odd that would have to do this.

Lastly, is physically deleting the index folders still the best way to reset the indexes?

Thanks again.

Gravatar

Josh Owens

November 5th 2007 00:00

Douglas,

I can't connect to: ./script/plugin install svn://svn.douglasfshearer.com/rails/plugins/acts_as_indexed

Will the plugin be available sometime soon?

Gravatar

Douglas F Shearer

November 5th 2007 09:33

Back up now.

Thanks for the heads up.

Gravatar

Mark

November 11th 2007 14:52

This sounds great, but I haven't been able to get it to download via RadRails. Every other plugin I've tried works well. Any ideas?

Gravatar

Mark

November 12th 2007 13:31

I figured out how to get it to download. You have to include the subversions bin files in the ruby path that RadRails is looking for.

Gravatar

paginator

November 17th 2007 12:50

Is it possible to combine this plugin with the paginating_find plugin to handle pagin?

I tried this, but it didn't work.

Post.find_with_index('my search query', :page => { :start => 1, :current => @page_number,:size => 1 })

This didn't work either

Post.find_with_index('my search query', find_options = {:page => { :start => 1, :current => @page_number,:size => 1 }})

Gravatar

sara

January 10th 2008 15:26

Hi,
ruby script/plugin install svn://svn.douglasfshearer.com/rails/plugins/acts_as_indexed

returns this message

svn: Can't connect to host 'svn.douglasfshearer.com': Connection refused

Is the svn down, or have i some local issue ?

Thanks.

Sara

Gravatar

Douglas F Shearer

January 10th 2008 16:22

Hi Sara.

SVN is down for the moment due to a server move. it should be back up tomorrow.

Status here: http://douglasfsh...

Gravatar

Douglas F Shearer

January 10th 2008 16:51

I have now added a downloadable version of the plugin above until SVN is restored.

Gravatar

sara

January 10th 2008 19:31

Thanks :) .

Gravatar

Andrew

February 4th 2008 22:51

When using :include with find_with_index, there is an ambiguous column error. This change seems to fix:

acts_as_indexed/lib/acts_as_indexed.rb, line 148:

<code> records = find(:all, :conditions => [ "#{class_name.tableize}.id IN (?)", part_query])</code>

Gravatar

Douglas F Shearer

February 5th 2008 00:16

Thanks for the heads up on that Andrew. Now fixed along with another ActiveRecord related error.

Gravatar

John Honovich

February 12th 2008 08:04

How does this plugin compare to act as sphinx?

I assume the lack of dependencies makes it easier to setup.

What are the relative shortcomings of acts_as_indexed versus using sphinx?

Thanks,

John

Gravatar

Douglas F Shearer

February 12th 2008 14:46

Hi John.

Shortcomings of acts_as_indexed are:
- Doesn't play nice with multiple physical servers.

And that's about it, other than maybe getting noticeably slow for searches with > 100, 000 records.

Sphinx, Ferret, Solr etc are all great in large environments, but they're a pain to setup and total overkill for most applications.

Gravatar

Vaqas

March 3rd 2008 13:23

Can I use 'boost' or something else to boost results from certain field index?

Gravatar

Douglas F Shearer

March 4th 2008 22:25

Hi Vaqas.

At the moment there is no way to weight a field in the way you are talking about. This is something I have on the feature list for the future.

Gravatar

Vaqas

March 21st 2008 08:38

What about wild cards?
~ * ?

Gravatar

Douglas F Shearer

March 21st 2008 10:34

Again, not a supported feature at the moment.

Gravatar

James

March 25th 2008 11:01

Hi.

Great plugin. Unfortunately, I need to search on a string of keywords logically connected by OR.

Before I dig into the code, any chance of supporting OR anytime soon?

----
The following query operators are supported:

* AND – This is the default option. ‘cat dog’ will find records matching ‘cat’ AND ‘dog’.
* NOT – ‘cat -dog’ will find records matching ‘cat’ AND NOT ‘dog’
* INCLUDE – ‘cat +me’ will find records matching ‘cat’ and ‘me’, even if ‘me’ is smaller than the min_word_size.
* ”” – Quoted terms are matched as phrases. ’”cat dog”’ will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator. ‘cat -“big dog”’ etc.
---

Gravatar

Sabrina

May 14th 2008 07:57

Hi, great plugin but I have a problem...I got a strange error

"Errno::ERANGE (Numerical result out of range - log):"

only in production mode.

I guess that the indexes get corrupted because I have this app where many users upload their restaurant infornmations and it could happen that two users submit at the same time.

Is there a way to not index the content when updating but to index periodically?

thanks

Gravatar

Sabrina

May 15th 2008 10:46

I've found where is the error.

acts_as_indexed/lib/search_atom.rb:93:in `log'

hope it's useful.

Gravatar

Douglas F Shearer

May 15th 2008 13:20

@Sabrina

I shall look into these bugs tomorrow.

Thanks for flagging them up!

Gravatar

Russ Jones

June 12th 2008 08:54

Any chance this works with single table inheritance (STI)?

Gravatar

Douglas F Shearer

June 12th 2008 10:08

Yes, it should work for STI, as long as it is called from the child classes.

Interesting question.

Gravatar

Mark Hurlburt

June 18th 2008 20:57

Is there any chance that you'll support partial word searches? The plugin is so easy to setup that its a shame that it doesn't seem to be that viable for searching on things like names where users are likely to not know the complete spelling and would like to limit results to entries that start with or contain a string fragment.

Gravatar

Douglas F Shearer

June 19th 2008 04:02

Hey Mark.

It wouldn't take much work, I don't think. Currently I'm only fixing critical bugs, but I plan to do a rewrite with new features in a few months.

Gravatar

Robb Shecter

July 2nd 2008 22:03

I just installed and tested it for the first time. Awesome job! I'd love to help contribute -- once it's up and running, I may look into partial word matches, etc.

Thanks again,
Robb

Add Your Comments


(Required)

Your email address to get your Gravatar. Address itself is not shown.

(Include the http://)

(Required)

 

You Are Here


Douglas F Shearer

This is the homepage of Douglas F Shearer, a software developer and mountainbike racer. Find out more at the About page.

Gallery Latest


Colin Left Malaysia Today Beverage In A Bag Malaysian Starbuck's Bears The Cows are Here Too! Cool Public Water Fountain Petronas Towers, Again

Stay Informed


What is RSS?

Top Tags