Investigating the search logic, which determines whether to index a given property of a given object.
The code currently extracts all the words from a value, then queries all the engines to see if they want the word list. I've modified the code to do a check through all the engines first, and if none will use the word list, it falls through and returns.
This avoids the server creating a word list for an enormous zip file or executable, and then throwing it away because no engines need it.
As far as the logic is concerned, the no_search_mimetypes can be used to exclude mime types from any space. It is a server-wide value, read in from the config file at startup.
The mimetypes configuration value in the 'search' group is used by each space to set up a list of acceptable mime-types to search. This is read in by the space at creation time (and can be modified as it's a standard list of string property)
Before this fix was implemented, mime-types that were allowed server wide (not in 'no_search_mimetypes') but not explicitly specified for a space, were still processed into word lists, and then rejected. For example, dropping a .exe file on a standard space would result in the server extracting all the words from the executable (timely if the file is of any size,) and then realise that non of the search engines for the space actually accept application/octet-stream mime-types. This has now been fixed.
Wednesday, October 04, 2006
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment