Douglas F Shearer

Rails Plugin: Acts_As_Indexed


Version 0.4.4 released 04 February 2008 – Fixed some minor AR bugs

This plugin allows ranked boolean-queried fulltext search to be added to any Rails app with no dependencies and minimal setup.

Install

./script/plugin install git://github.com/dougal/acts_as_indexed.git

If you don’t have git installed, you can download the plugin from the GitHub page and unpack it into the vendor/plugins directory of your rails app.

Setup

Add acts_as_indexed to the top of any models you want to index, along with a list of the fields you wish to be indexed.


class Post < ActiveRecord::Base
  acts_as_indexed :fields => [:title, :body]

   ...
end

Searching

To search, call the find_with_index method on your model to search using the index. The optional ids_only parameter, when set to true, will return only the IDs of any matching records.


    # Returns array of Post objects.
	my_search_results = Post.find_with_index('my search query') # =>  [#<Post:0x314b09c @attributes={"...
	
	# Pass any of the ActiveRecord find options to the search.
	my_search_results = Post.find_with_index('my search query',{:limit => 10}) # return the first 10 matches.
	
	# Returns array of IDs.
	my_search_results = Post.find_with_index('my search query',{},{:ids_only => true}) # =>  [12,19,33...

h4. Boolean Query Options

The following query operators are supported:

  • AND – This is the default option. ‘cat dog’ will find records matching ‘cat’ AND ‘dog’.
  • NOT – ‘cat -dog’ will find records matching ‘cat’ AND NOT ‘dog’
  • INCLUDE – ‘cat +me’ will find records matching ‘cat’ and ‘me’, even if ‘me’ is smaller than the min_word_size.
  • "" – Quoted terms are matched as phrases. ‘“cat dog”’ will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator. ‘cat -“big dog”’ etc.

Pagination

Pagination is supported via the paginate_search method whose first argument is the search query, followed all the standard will_paginate arguments.


@images = Image.paginate_search 'girl', :page => 1, :per_page => 5

Other Stuff

Full Documentation

You can either build the rdoc by running rake rdoc in the acts_as_indexed directory, or look at the latest version online.

Problems, Comments, Suggestions?

All of the above are most welcome. dougal.s@gmail.com

Credits

Douglas F Shearer

Donate

If you find this plugin useful, please consider a "donation (Paypal) ":https://www.paypal.com/cgi-bin/webscr?cmd=xclick&business=dougal%2es%40gmail%2ecom&item_name=Douglas%20F%20Shearer&buyer_credit_promo_code=&buyer_credit_product_category=&buyer_credit_shipping_method=&buyer_credit_user_address_change=&page_style=Primary&no_shipping=0&return=http%3a%2f%2fdouglasfshearer%2ecom&cancel_return=http%3a%2f%2fdouglasfshearer%2ecom&no_note=1&cn=I%20am%20donating%20because%2e%2e%2e&tax=0&currencycode=GBP&lc=GB&bn=PP%2dDonationsBF&charset=UTF%2d8 to show your support!

 

Comments


Gravatar

Jim

September 12 2007 22:29

Very promising plugin.

Are you able to index cross-models?

For example, if my "Post" model belonged to a "Category" model, could I index on the category_name field?

Gravatar

Jim

September 12 2007 23:44

I just realized that I can do what I was asking about via an instance method.

So in the above example, the method would be:

******************
Class Post < ActiveRecord::Base

acts_as_indexed :fields => [:name, :post_category_name]

belongs_to :category

def post_category_name
category.name
end

end
******************

Very cool.

Next question. Is there a way to reset the index other than deleting the "index" folder?

Gravatar

Douglas F Shearer

September 13 2007 09:41

Hi Jim.

Glad you figured out your own query, having instance methods makes a lot of things easy.

ATM there is no way to reset the index other than to delete it. Obviously filesystem access is not always available, so the next version will have a method to do this from within an application (If you look carefully in the source, there has been a stub for the method since the first version, just haven't got round to doing it).

Gravatar

Jim

September 13 2007 16:56

Hi Douglas,

Thanks for the response. I have some more feedback after playing with the plugin some more this morning. If you'd rather chat through email versus blogs comments, let me know.

Feedback #1:
NOT search terms are modified when find_with_index is called. If I want to make two consecutive find_with_index calls on two different models, the second call is missing the NOT term.

Example of what I was trying to do below. The params SHOULD be the same in both cases.

**********************
#params[:q] => "cooking -japanese"
l = Lesson.find_with_index params[:q]

#params[:q] => "cooking "
s = Suggestion.find_with_index params[:q]
**********************

Feedback #2:
If my search has two terms and one of the terms is junk data, all of the results with the first term are returned. I would think nothing would be returned in this case.

For example, if I searched for 'dog asdfasdfas' (not quoted) all records with dog would be returned even though 'asdfasdfas' does not exist in the database.

Please feel free to email me if you have any questions. [jim at praexis dot com]

Gravatar

Douglas F Shearer

September 14 2007 10:01

Hi again Jim.

I'll drop you an email, but as a brief response..

#1 Modified query param - That's a bug, will fix that just now.

#2 Default action for a space character is AND rather than OR. I can see what you mean about having it return 'dog' if 'asdfasdfas' doesn't exist at all, but most people would consider that to be unexpected behavior.

Gravatar

Raul Galindo

October 05 2007 12:07

Hi Jim
I cant reach the address the plugin is in
is this because the server is down or am i doing something wrong?
i wait for ur reply
Best regards.

Gravatar

Douglas F Shearer

October 05 2007 12:25

Hi Raul.

The svn server is working fine for me. No reason it should be down as it's hosted on the same machine as this site.

The plugin install command above should work in most cases, but if for some reason your permissions are wrong you may need to run it as..

ruby script/plugin

rather than

./script/plugin

Hope this helps.

Gravatar

cavatina

October 07 2007 21:35

hi

svn server is not working

Gravatar

Douglas F Shearer

October 08 2007 11:32

Working fine for me.

The svn address above WILL NOT work in a browser.

I've realised this is a problem for windows users without svn, so I may move to Apache Web-DAV or provide an archived download.

Gravatar

Jim

October 16 2007 16:03

Hi Douglas,

I updated to your latest code yesterday. Nice improvements on the quoted searches and the relevance rankings. Nice work!

With that being said, I'm seeing a couple of issues when running tests.

Error #1: Errno::EDOM: Numerical argument out of domain - log
- Deleting the indexes and rerunning th e tests seems to fix the issue, but it has come back a few times.

Error #2: A file name "size" does not exist the first time I run the test with my existing indexes. Deleting the indexes and rerunning also fixes this error, but it seems odd that would have to do this.

Lastly, is physically deleting the index folders still the best way to reset the indexes?

Thanks again.

Gravatar

Josh Owens

November 05 2007 00:00

Douglas,

I can't connect to: ./script/plugin install svn://svn.douglasfshearer.com/rails/plugins/acts_as_indexed

Will the plugin be available sometime soon?

Gravatar

Douglas F Shearer

November 05 2007 09:33

Back up now.

Thanks for the heads up.

Gravatar

Mark

November 11 2007 14:52

This sounds great, but I haven't been able to get it to download via RadRails. Every other plugin I've tried works well. Any ideas?

Gravatar

Mark

November 12 2007 13:31

I figured out how to get it to download. You have to include the subversions bin files in the ruby path that RadRails is looking for.

Gravatar

paginator

November 17 2007 12:50

Is it possible to combine this plugin with the paginating_find plugin to handle pagin?

I tried this, but it didn't work.

Post.find_with_index('my search query', :page => { :start => 1, :current => @page_number,:size => 1 })

This didn't work either

Post.find_with_index('my search query', find_options = {:page => { :start => 1, :current => @page_number,:size => 1 }})

Gravatar

sara

January 10 2008 15:26

Hi,
ruby script/plugin install svn://svn.douglasfshearer.com/rails/plugins/acts_as_indexed

returns this message

svn: Can't connect to host 'svn.douglasfshearer.com': Connection refused

Is the svn down, or have i some local issue ?

Thanks.

Sara

Gravatar

Douglas F Shearer

January 10 2008 16:22

Hi Sara.

SVN is down for the moment due to a server move. it should be back up tomorrow.

Status here: http://douglasfsh...

Gravatar

Douglas F Shearer

January 10 2008 16:51

I have now added a downloadable version of the plugin above until SVN is restored.

Gravatar

sara

January 10 2008 19:31

Thanks :) .

Gravatar

Andrew

February 04 2008 22:51

When using :include with find_with_index, there is an ambiguous column error. This change seems to fix:

acts_as_indexed/lib/acts_as_indexed.rb, line 148:

<code> records = find(:all, :conditions => [ "#{class_name.tableize}.id IN (?)", part_query])</code>

Gravatar

Douglas F Shearer

February 05 2008 00:16

Thanks for the heads up on that Andrew. Now fixed along with another ActiveRecord related error.

Gravatar

John Honovich

February 12 2008 08:04

How does this plugin compare to act as sphinx?

I assume the lack of dependencies makes it easier to setup.

What are the relative shortcomings of acts_as_indexed versus using sphinx?

Thanks,

John

Gravatar

Douglas F Shearer

February 12 2008 14:46

Hi John.

Shortcomings of acts_as_indexed are:
- Doesn't play nice with multiple physical servers.

And that's about it, other than maybe getting noticeably slow for searches with > 100, 000 records.

Sphinx, Ferret, Solr etc are all great in large environments, but they're a pain to setup and total overkill for most applications.

Gravatar

Vaqas

March 03 2008 13:23

Can I use 'boost' or something else to boost results from certain field index?

Gravatar

Douglas F Shearer

March 04 2008 22:25

Hi Vaqas.

At the moment there is no way to weight a field in the way you are talking about. This is something I have on the feature list for the future.

Gravatar

Vaqas

March 21 2008 08:38

What about wild cards?
~ * ?

Gravatar

Douglas F Shearer

March 21 2008 10:34

Again, not a supported feature at the moment.

Gravatar

James

March 25 2008 11:01

Hi.

Great plugin. Unfortunately, I need to search on a string of keywords logically connected by OR.

Before I dig into the code, any chance of supporting OR anytime soon?

----
The following query operators are supported:

* AND – This is the default option. ‘cat dog’ will find records matching ‘cat’ AND ‘dog’.
* NOT – ‘cat -dog’ will find records matching ‘cat’ AND NOT ‘dog’
* INCLUDE – ‘cat +me’ will find records matching ‘cat’ and ‘me’, even if ‘me’ is smaller than the min_word_size.
* ”” – Quoted terms are matched as phrases. ’”cat dog”’ will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator. ‘cat -“big dog”’ etc.
---

Gravatar

Sabrina

May 14 2008 07:57

Hi, great plugin but I have a problem...I got a strange error

"Errno::ERANGE (Numerical result out of range - log):"

only in production mode.

I guess that the indexes get corrupted because I have this app where many users upload their restaurant infornmations and it could happen that two users submit at the same time.

Is there a way to not index the content when updating but to index periodically?

thanks

Gravatar

Sabrina

May 15 2008 10:46

I've found where is the error.

acts_as_indexed/lib/search_atom.rb:93:in `log'

hope it's useful.

Gravatar

Douglas F Shearer

May 15 2008 13:20

@Sabrina

I shall look into these bugs tomorrow.

Thanks for flagging them up!

Gravatar

Russ Jones

June 12 2008 08:54

Any chance this works with single table inheritance (STI)?

Gravatar

Douglas F Shearer

June 12 2008 10:08

Yes, it should work for STI, as long as it is called from the child classes.

Interesting question.

Gravatar

Mark Hurlburt

June 18 2008 20:57

Is there any chance that you'll support partial word searches? The plugin is so easy to setup that its a shame that it doesn't seem to be that viable for searching on things like names where users are likely to not know the complete spelling and would like to limit results to entries that start with or contain a string fragment.

Gravatar

Douglas F Shearer

June 19 2008 04:02

Hey Mark.

It wouldn't take much work, I don't think. Currently I'm only fixing critical bugs, but I plan to do a rewrite with new features in a few months.

Gravatar

Robb Shecter

July 02 2008 22:03

I just installed and tested it for the first time. Awesome job! I'd love to help contribute -- once it's up and running, I may look into partial word matches, etc.

Thanks again,
Robb

Gravatar

Matt

July 17 2008 20:14

Will you be moving this to GitHub any time soon?

Gravatar

Douglas F Shearer

July 21 2008 08:01

Hey Matt.

Since script/plugin now supports git for plugin installs, the plan is to move to github this week or next.

I would have done this earlier, but have been away from a computer for the last few weeks.

It shall be soon!

Gravatar

Sharon

July 28 2008 21:55

Thanks for the plugin. I had the same question as James above, though - is there any possibility of support for OR as well as AND boolean queries?

Gravatar

Sebad

October 18 2008 01:50

Thanks for the plugin. it is awesome.
But just after I move from rails 2.0 to 2.1.
I had the following error

NoMethodError (undefined method `acts_as_indexed' for #<Class:0x471bed0>):
C:/Desarrollos/ruby/lib/ruby/gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:1672:in `method_missing_without_paginate'
/vendor/plugins/will_paginate/lib/will_paginate/finder.rb:167:in `method_missing'

I have the plugins in
\vendor\plugins\will_paginate
\vendor\plugins\acts_as_indexed

Do you know waht is this error?
Thanks in advance.

Gravatar

Dan Sweeney

September 23 2008 20:19

Hi, great plugin! I'd like to second the request for partial word searches from a few months back... any progress on that? I'd love to help i'm afraid my skill set isn't quite there though.

Gravatar

Michail

January 13 2009 13:41

Thanks for the plugin!
Can it do search in several models at once?
It's need for paginating result of common searching

Gravatar

Douglas F Shearer

January 13 2009 20:02

Hi Michael.

I'm afraid at the moment that is not possible. Good idea for an addition though, I'll think about it over the next few days.

Gravatar

Michail

January 14 2009 10:30

Hi Douglas.
Thank you, it will be good stuff! :)

But now I've done little overhead using will_paginate plugin.

http://pastie.org...
It's simple and not so clear, but maybe somebody will find it useful.

Gravatar

Michail

February 26 2009 07:16

Hi Douglas!
Please, help me with one issue.
How can I do case independent search with acts_as_indexed?

Gravatar

Douglas F Shearer

February 26 2009 12:16

To make the plugin case-sensitive, remove the .downcase method-call from line 304 of lib/search_index.rb.

I have this pencilled in as a configurable in a future version.

Gravatar

Michail

February 26 2009 16:27

Thank you!

Gravatar

Michail

February 26 2009 17:15

Douglas, problem was in non latin chars.
So i fixed it in that line 304
atoms = s.mb_chars.downcase.to_s.gsub(/\W/,' ').squeeze(' ').split

That's why:
"Михаил".downcase
=> "Михаил"
but
"Михаил".mb_chars.downcase.to_s
=> "михаил".

So thank you for advice!

Gravatar

Douglas F Shearer

February 26 2009 22:44

Thanks for the feedback Michail, much appreciated. The next release will be UTF8 compatible, as well as Ruby 1.9 compatible.

Gravatar

Kostas Papadimakis

June 10 2009 13:44

Hi Douglas,

Great plugin, I already used it for a new application. Unfortunately, I tested with Ruby 1.8.7 and when I uploaded my app to a hosting provider I found out that Ruby 1.8.6 does not support the ord method for Fixnum. They can't change Ruby versions, so I thought I might share the problem and the solution I found with anyone interested ..

So, in your search_index.rb, you use the ord method for atom[0] (which results in a Fixnum). The solution I found was a quick hack adding an "ord" method to Fixnum class, which returns "self"

I hope this does not lead to new bugs :)

Have a nice day

def encoded_prefix(atom)
prefix = atom[0,@index_depth]
if !@prefix_cache || !@prefix_cache.has_key?(prefix)
@prefix_cache = {} if !@prefix_cache
len = atom.length
if len > 1
@prefix_cache[prefix] = prefix.split(//).collect{|c| c[0]}.inject{|sum,c| sum.to_s + '_' + c.to_s}
else
@prefix_cache[prefix] = atom[0].ord.to_s
end
end
@prefix_cache[prefix]
end

Gravatar

einar

June 11 2009 18:12

Hi thanks for the plugin.
I get this error when i save a record with a textfield that uses MCE-Editor-Options for saving the text with formatting options. If i leave the datafield out in the acts_as_indexed :fields =>... then all other textfields are saved and everything is working.
Any idea why?
tia

Gravatar

einar

June 11 2009 18:13

Sorry here ist the errormessage:
RAILS_ROOT: D:/INSTAN~1/rails_apps/ekdb
Application Trace | Framework Trace | Full Trace

vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:187:in `encoded_prefix'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:285:in `load_atoms'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:285:in `collect'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:285:in `load_atoms'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:60:in `update_record'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/acts_as_indexed.rb:97:in `index_update'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/acts_as_indexed.rb:211:in `update_index'
D:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/callbacks.rb:307:in `send'
D:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/callbacks.rb:307:in `callback'

Gravatar

einar

June 11 2009 18:14

oops was truncated: the last message is:

undefined method `ord' for 112:Fixnum

Gravatar

einar

June 11 2009 18:23

hi kostas, that seems to be my problem. You forgot to post the "hacked code" i think. Can you do so? And where do i have to apply the hacked code?

Gravatar

Douglas F Shearer

June 11 2009 19:43

Kostas and Einar - Thanks for your bug reports. 1.8.6 compatibility has now been fixed in my latest release.

Gravatar

Kostas Papadimakis

June 11 2009 20:34

# quick hack for acts_as_indexed on ruby 1.8.6
class Fixnum
def ord
self
end
end

I added this code in app/helpers/application_helper.rb (outside of the ApplicationHelper module) since I didn't know where else to put it. I'm not sure that Ruby returns the number when calling ord on a Fixnum, though.

If someone knows for sure, I would be glad to hear :)

Gravatar

einar

June 12 2009 05:38

thanks douglas, it now works fine for me.
Question:
How can i exclude some fields for one search? And is it possible to search for substrings?
i.E. its raining and i type in "rain" and raining will be found?

Gravatar

Douglas F Shearer

June 12 2009 09:09

Kostas - The fix I put in place avoids having to modify Ruby Core classes, and instead probes for support of the ord method. See http://github.com....

Einar - Exclusion of certain fields, and substring search are items I have on my list of items to support in future versions. I couldn't, with any accuracy, say when these would be available.

Gravatar

Benjamin K

June 13 2009 18:59

Hi Douglas. I'm having a little trouble with Acts_As_Index missing some records. I have encountered several cases where find_with_index does not return a matching record. In every case I have observed that the query has been an entire searchable field (e.g. find_with_index('dogs') should return a record with an indexed field of 'dogs'), although find_with_index certainly doesn't fail for all such queries.

I have tried deleting the index files to force a rebuild, but the same records are missed each time.

I would be happy to send you a copy of my index files and some sample failing queries if this will help you track down the bug.

Gravatar

Douglas F Shearer

June 17 2009 20:38

Hi Benjamin.

I've replied to you via email. Issue can be tracked here: http://github.com...

Thanks.

Gravatar

booogy

March 06 2010 03:46

http://www.corren... discount cialis 759996 http://www.produc... internet pharmacy propecia soma zoloft %-O http://www.santil... tramadol hcl msiig

Gravatar

uvela

March 06 2010 07:24

http://www.jeumpa... valium online 8-[[[ http://www.produc... aciphex 8-]]] http://www.bookam... buy xanax jnn http://www.bowlin... buy doxycycline ashrm

Gravatar

quicklink

March 08 2010 00:00

http://www.perrys... generic xanax ueqpc http://www.paesin... buy acomplia rimonabant online 8D http://www.ahssa.... retin 9422 http://www.kawtal... ultram %DD http://www.produc... accutane 6198

Gravatar

mybutton

March 08 2010 02:50

http://www.bowlin... prednisone 404 http://www.corren... cialis on line ubeh http://www.paesin... ambien medication igndss http://www.produc... generic propecia 54911

Gravatar

comanche

March 09 2010 23:15

http://www.buytra... buy tramadol us pharmacies 499652 http://www.yourba... xanax >:-DDD http://www.karens... ultracet 6671 http://www.odoat.... buy prednisone %]

Gravatar

roksana

March 10 2010 09:53

http://www.buytra... tramadol 200 pills online unbgyj http://www.raefra... accutane npaas http://www.bedava... aciphex :-[[[ http://www.odoat.... accutane snqcy

Add Your Comments


(Required)

Your email address to get your Gravatar. Address itself is not shown.

(Include the http://)

(Required)

 

You Are Here


Douglas F Shearer

This is the homepage of Douglas F Shearer, a software developer and mountainbike racer. More…

Hire Me!


I'm available for hire. Ruby, Java and PHP work, both remotely (Worldwide) and locally (Scotland). Find out more or email me.

Flickr Latest


Stay Informed


What is RSS?

Categories


  1. Bike (90)
  2. Coding (84)
  3. Other (46)

Top Tags