Version 0.4.4 released 04 February 2008 – Fixed some minor AR bugs
This plugin allows ranked boolean-queried fulltext search to be added to any Rails app with no dependencies and minimal setup.
Install
./script/plugin install git://github.com/dougal/acts_as_indexed.git
If you don’t have git installed, you can download the plugin from the GitHub page and unpack it into the vendor/plugins directory of your rails app.
Setup
Add acts_as_indexed to the top of any models you want to index, along with a list of the fields you wish to be indexed.
class Post < ActiveRecord::Base
acts_as_indexed :fields => [:title, :body]
...
end
Searching
To search, call the find_with_index method on your model to search using the index. The optional ids_only parameter, when set to true, will return only the IDs of any matching records.
# Returns array of Post objects.
my_search_results = Post.find_with_index('my search query') # => [#<Post:0x314b09c @attributes={"...
# Pass any of the ActiveRecord find options to the search.
my_search_results = Post.find_with_index('my search query',{:limit => 10}) # return the first 10 matches.
# Returns array of IDs.
my_search_results = Post.find_with_index('my search query',{},{:ids_only => true}) # => [12,19,33...
Boolean Query Options
The following query operators are supported:
- AND – This is the default option. ‘cat dog’ will find records matching ‘cat’ AND ‘dog’.
- NOT – ‘cat -dog’ will find records matching ‘cat’ AND NOT ‘dog’
- INCLUDE – ‘cat +me’ will find records matching ‘cat’ and ‘me’, even if ‘me’ is smaller than the
min_word_size.
- ”” – Quoted terms are matched as phrases. ’”cat dog”’ will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator. ‘cat -“big dog”’ etc.
Pagination
Pagination is supported via the paginate_search method whose first argument is the search query, followed all the standard will_paginate arguments.
@images = Image.paginate_search 'girl', :page => 1, :per_page => 5
Other Stuff
Full Documentation
You can either build the rdoc by running rake rdoc in the acts_as_indexed directory, or look at the latest version online.
Problems, Comments, Suggestions?
All of the above are most welcome. dougal.s@gmail.com
Credits
Douglas F Shearer
Donate
If you find this plugin useful, please consider a donation (Paypal) to show your support!
Tags
Ruby on rails, Plugin, Fulltext search, Search, Index, Acts_as_indexed.
August 30 2007 15:00 |
Posted in Coding |
comments (60)
September 12 2007 22:29
Very promising plugin.
Are you able to index cross-models?
For example, if my "Post" model belonged to a "Category" model, could I index on the category_name field?
September 12 2007 23:44
I just realized that I can do what I was asking about via an instance method.
So in the above example, the method would be:
******************
Class Post < ActiveRecord::Base
acts_as_indexed :fields => [:name, :post_category_name]
belongs_to :category
def post_category_name
category.name
end
end
******************
Very cool.
Next question. Is there a way to reset the index other than deleting the "index" folder?
September 13 2007 09:41
Hi Jim.
Glad you figured out your own query, having instance methods makes a lot of things easy.
ATM there is no way to reset the index other than to delete it. Obviously filesystem access is not always available, so the next version will have a method to do this from within an application (If you look carefully in the source, there has been a stub for the method since the first version, just haven't got round to doing it).
September 13 2007 16:56
Hi Douglas,
Thanks for the response. I have some more feedback after playing with the plugin some more this morning. If you'd rather chat through email versus blogs comments, let me know.
Feedback #1:
NOT search terms are modified when find_with_index is called. If I want to make two consecutive find_with_index calls on two different models, the second call is missing the NOT term.
Example of what I was trying to do below. The params SHOULD be the same in both cases.
**********************
#params[:q] => "cooking -japanese"
l = Lesson.find_with_index params[:q]
#params[:q] => "cooking "
s = Suggestion.find_with_index params[:q]
**********************
Feedback #2:
If my search has two terms and one of the terms is junk data, all of the results with the first term are returned. I would think nothing would be returned in this case.
For example, if I searched for 'dog asdfasdfas' (not quoted) all records with dog would be returned even though 'asdfasdfas' does not exist in the database.
Please feel free to email me if you have any questions. [jim at praexis dot com]
September 14 2007 10:01
Hi again Jim.
I'll drop you an email, but as a brief response..
#1 Modified query param - That's a bug, will fix that just now.
#2 Default action for a space character is AND rather than OR. I can see what you mean about having it return 'dog' if 'asdfasdfas' doesn't exist at all, but most people would consider that to be unexpected behavior.
October 05 2007 12:07
Hi Jim
I cant reach the address the plugin is in
is this because the server is down or am i doing something wrong?
i wait for ur reply
Best regards.
October 05 2007 12:25
Hi Raul.
The svn server is working fine for me. No reason it should be down as it's hosted on the same machine as this site.
The plugin install command above should work in most cases, but if for some reason your permissions are wrong you may need to run it as..
ruby script/plugin
rather than
./script/plugin
Hope this helps.
October 07 2007 21:35
hi
svn server is not working
October 08 2007 11:32
Working fine for me.
The svn address above WILL NOT work in a browser.
I've realised this is a problem for windows users without svn, so I may move to Apache Web-DAV or provide an archived download.
October 16 2007 16:03
Hi Douglas,
I updated to your latest code yesterday. Nice improvements on the quoted searches and the relevance rankings. Nice work!
With that being said, I'm seeing a couple of issues when running tests.
Error #1: Errno::EDOM: Numerical argument out of domain - log
- Deleting the indexes and rerunning th e tests seems to fix the issue, but it has come back a few times.
Error #2: A file name "size" does not exist the first time I run the test with my existing indexes. Deleting the indexes and rerunning also fixes this error, but it seems odd that would have to do this.
Lastly, is physically deleting the index folders still the best way to reset the indexes?
Thanks again.
November 05 2007 00:00
Douglas,
I can't connect to: ./script/plugin install svn://svn.douglasfshearer.com/rails/plugins/acts_as_indexed
Will the plugin be available sometime soon?
November 05 2007 09:33
Back up now.
Thanks for the heads up.
November 11 2007 14:52
This sounds great, but I haven't been able to get it to download via RadRails. Every other plugin I've tried works well. Any ideas?
November 12 2007 13:31
I figured out how to get it to download. You have to include the subversions bin files in the ruby path that RadRails is looking for.
November 17 2007 12:50
Is it possible to combine this plugin with the paginating_find plugin to handle pagin?
I tried this, but it didn't work.
Post.find_with_index('my search query', :page => { :start => 1, :current => @page_number,:size => 1 })
This didn't work either
Post.find_with_index('my search query', find_options = {:page => { :start => 1, :current => @page_number,:size => 1 }})
January 10 2008 15:26
Hi,
ruby script/plugin install svn://svn.douglasfshearer.com/rails/plugins/acts_as_indexed
returns this message
svn: Can't connect to host 'svn.douglasfshearer.com': Connection refused
Is the svn down, or have i some local issue ?
Thanks.
Sara
January 10 2008 16:22
Hi Sara.
SVN is down for the moment due to a server move. it should be back up tomorrow.
Status here: http://douglasfsh...
January 10 2008 16:51
I have now added a downloadable version of the plugin above until SVN is restored.
January 10 2008 19:31
Thanks :) .
February 04 2008 22:51
When using :include with find_with_index, there is an ambiguous column error. This change seems to fix:
acts_as_indexed/lib/acts_as_indexed.rb, line 148:
<code> records = find(:all, :conditions => [ "#{class_name.tableize}.id IN (?)", part_query])</code>
February 05 2008 00:16
Thanks for the heads up on that Andrew. Now fixed along with another ActiveRecord related error.
February 12 2008 08:04
How does this plugin compare to act as sphinx?
I assume the lack of dependencies makes it easier to setup.
What are the relative shortcomings of acts_as_indexed versus using sphinx?
Thanks,
John
February 12 2008 14:46
Hi John.
Shortcomings of acts_as_indexed are:
- Doesn't play nice with multiple physical servers.
And that's about it, other than maybe getting noticeably slow for searches with > 100, 000 records.
Sphinx, Ferret, Solr etc are all great in large environments, but they're a pain to setup and total overkill for most applications.
March 03 2008 13:23
Can I use 'boost' or something else to boost results from certain field index?
March 04 2008 22:25
Hi Vaqas.
At the moment there is no way to weight a field in the way you are talking about. This is something I have on the feature list for the future.
March 21 2008 08:38
What about wild cards?
~ * ?
March 21 2008 10:34
Again, not a supported feature at the moment.
March 25 2008 11:01
Hi.
Great plugin. Unfortunately, I need to search on a string of keywords logically connected by OR.
Before I dig into the code, any chance of supporting OR anytime soon?
----
The following query operators are supported:
* AND – This is the default option. ‘cat dog’ will find records matching ‘cat’ AND ‘dog’.
* NOT – ‘cat -dog’ will find records matching ‘cat’ AND NOT ‘dog’
* INCLUDE – ‘cat +me’ will find records matching ‘cat’ and ‘me’, even if ‘me’ is smaller than the min_word_size.
* ”” – Quoted terms are matched as phrases. ’”cat dog”’ will find records matching the whole phrase. Quoted terms can be preceded by the NOT operator. ‘cat -“big dog”’ etc.
---
May 14 2008 07:57
Hi, great plugin but I have a problem...I got a strange error
"Errno::ERANGE (Numerical result out of range - log):"
only in production mode.
I guess that the indexes get corrupted because I have this app where many users upload their restaurant infornmations and it could happen that two users submit at the same time.
Is there a way to not index the content when updating but to index periodically?
thanks
May 15 2008 10:46
I've found where is the error.
acts_as_indexed/lib/search_atom.rb:93:in `log'
hope it's useful.
May 15 2008 13:20
@Sabrina
I shall look into these bugs tomorrow.
Thanks for flagging them up!
June 12 2008 08:54
Any chance this works with single table inheritance (STI)?
June 12 2008 10:08
Yes, it should work for STI, as long as it is called from the child classes.
Interesting question.
June 18 2008 20:57
Is there any chance that you'll support partial word searches? The plugin is so easy to setup that its a shame that it doesn't seem to be that viable for searching on things like names where users are likely to not know the complete spelling and would like to limit results to entries that start with or contain a string fragment.
June 19 2008 04:02
Hey Mark.
It wouldn't take much work, I don't think. Currently I'm only fixing critical bugs, but I plan to do a rewrite with new features in a few months.
July 02 2008 22:03
I just installed and tested it for the first time. Awesome job! I'd love to help contribute -- once it's up and running, I may look into partial word matches, etc.
Thanks again,
Robb
July 17 2008 20:14
Will you be moving this to GitHub any time soon?
July 21 2008 08:01
Hey Matt.
Since script/plugin now supports git for plugin installs, the plan is to move to github this week or next.
I would have done this earlier, but have been away from a computer for the last few weeks.
It shall be soon!
July 28 2008 21:55
Thanks for the plugin. I had the same question as James above, though - is there any possibility of support for OR as well as AND boolean queries?
October 18 2008 01:50
Thanks for the plugin. it is awesome.
But just after I move from rails 2.0 to 2.1.
I had the following error
NoMethodError (undefined method `acts_as_indexed' for #<Class:0x471bed0>):
C:/Desarrollos/ruby/lib/ruby/gems/1.8/gems/activerecord-2.1.1/lib/active_record/base.rb:1672:in `method_missing_without_paginate'
/vendor/plugins/will_paginate/lib/will_paginate/finder.rb:167:in `method_missing'
I have the plugins in
\vendor\plugins\will_paginate
\vendor\plugins\acts_as_indexed
Do you know waht is this error?
Thanks in advance.
September 23 2008 20:19
Hi, great plugin! I'd like to second the request for partial word searches from a few months back... any progress on that? I'd love to help i'm afraid my skill set isn't quite there though.
January 13 2009 13:41
Thanks for the plugin!
Can it do search in several models at once?
It's need for paginating result of common searching
January 13 2009 20:02
Hi Michael.
I'm afraid at the moment that is not possible. Good idea for an addition though, I'll think about it over the next few days.
January 14 2009 10:30
Hi Douglas.
Thank you, it will be good stuff! :)
But now I've done little overhead using will_paginate plugin.
http://pastie.org...
It's simple and not so clear, but maybe somebody will find it useful.
February 26 2009 07:16
Hi Douglas!
Please, help me with one issue.
How can I do case independent search with acts_as_indexed?
February 26 2009 12:16
To make the plugin case-sensitive, remove the .downcase method-call from line 304 of lib/search_index.rb.
I have this pencilled in as a configurable in a future version.
February 26 2009 16:27
Thank you!
February 26 2009 17:15
Douglas, problem was in non latin chars.
So i fixed it in that line 304
atoms = s.mb_chars.downcase.to_s.gsub(/\W/,' ').squeeze(' ').split
That's why:
"Михаил".downcase
=> "Михаил"
but
"Михаил".mb_chars.downcase.to_s
=> "михаил".
So thank you for advice!
February 26 2009 22:44
Thanks for the feedback Michail, much appreciated. The next release will be UTF8 compatible, as well as Ruby 1.9 compatible.
June 10 2009 13:44
Hi Douglas,
Great plugin, I already used it for a new application. Unfortunately, I tested with Ruby 1.8.7 and when I uploaded my app to a hosting provider I found out that Ruby 1.8.6 does not support the ord method for Fixnum. They can't change Ruby versions, so I thought I might share the problem and the solution I found with anyone interested ..
So, in your search_index.rb, you use the ord method for atom[0] (which results in a Fixnum). The solution I found was a quick hack adding an "ord" method to Fixnum class, which returns "self"
I hope this does not lead to new bugs :)
Have a nice day
def encoded_prefix(atom)
prefix = atom[0,@index_depth]
if !@prefix_cache || !@prefix_cache.has_key?(prefix)
@prefix_cache = {} if !@prefix_cache
len = atom.length
if len > 1
@prefix_cache[prefix] = prefix.split(//).collect{|c| c[0]}.inject{|sum,c| sum.to_s + '_' + c.to_s}
else
@prefix_cache[prefix] = atom[0].ord.to_s
end
end
@prefix_cache[prefix]
end
June 11 2009 18:12
Hi thanks for the plugin.
I get this error when i save a record with a textfield that uses MCE-Editor-Options for saving the text with formatting options. If i leave the datafield out in the acts_as_indexed :fields =>... then all other textfields are saved and everything is working.
Any idea why?
tia
June 11 2009 18:13
Sorry here ist the errormessage:
RAILS_ROOT: D:/INSTAN~1/rails_apps/ekdb
Application Trace | Framework Trace | Full Trace
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:187:in `encoded_prefix'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:285:in `load_atoms'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:285:in `collect'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:285:in `load_atoms'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/search_index.rb:60:in `update_record'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/acts_as_indexed.rb:97:in `index_update'
vendor/plugins/acts_as_indexed/dougal-acts_as_indexed-14f0c8baf4a6fb799f14a1aa175877444eb3e6e5/lib/acts_as_indexed.rb:211:in `update_index'
D:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/callbacks.rb:307:in `send'
D:/INSTAN~1/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/callbacks.rb:307:in `callback'
June 11 2009 18:14
oops was truncated: the last message is:
undefined method `ord' for 112:Fixnum
June 11 2009 18:23
hi kostas, that seems to be my problem. You forgot to post the "hacked code" i think. Can you do so? And where do i have to apply the hacked code?
June 11 2009 19:43
Kostas and Einar - Thanks for your bug reports. 1.8.6 compatibility has now been fixed in my latest release.
June 11 2009 20:34
# quick hack for acts_as_indexed on ruby 1.8.6
class Fixnum
def ord
self
end
end
I added this code in app/helpers/application_helper.rb (outside of the ApplicationHelper module) since I didn't know where else to put it. I'm not sure that Ruby returns the number when calling ord on a Fixnum, though.
If someone knows for sure, I would be glad to hear :)
June 12 2009 05:38
thanks douglas, it now works fine for me.
Question:
How can i exclude some fields for one search? And is it possible to search for substrings?
i.E. its raining and i type in "rain" and raining will be found?
June 12 2009 09:09
Kostas - The fix I put in place avoids having to modify Ruby Core classes, and instead probes for support of the ord method. See http://github.com....
Einar - Exclusion of certain fields, and substring search are items I have on my list of items to support in future versions. I couldn't, with any accuracy, say when these would be available.
June 13 2009 18:59
Hi Douglas. I'm having a little trouble with Acts_As_Index missing some records. I have encountered several cases where find_with_index does not return a matching record. In every case I have observed that the query has been an entire searchable field (e.g. find_with_index('dogs') should return a record with an indexed field of 'dogs'), although find_with_index certainly doesn't fail for all such queries.
I have tried deleting the index files to force a rebuild, but the same records are missed each time.
I would be happy to send you a copy of my index files and some sample failing queries if this will help you track down the bug.
June 17 2009 20:38
Hi Benjamin.
I've replied to you via email. Issue can be tracked here: http://github.com...
Thanks.
Add Your Comments
