Wikia Gets Closer to Launching - A Grubby Solution?
(Page 2 of 4 )
Search engines index the Internet and deliver relevant results to users in response to key words. At first, it seemed as if Wales planned to use volunteers to handle this, completely replacing the algorithm. But he found a better answer.
LookSmart purchased a search project that it later neglected, named Grub. Wikia came in and bought it, and made it open source for the first time in four years. Grub is an Internet indexing program; the fact that it is now open source means that programmers can work on the code, modify it, test it, and in general build something better.
That transparency is a key aspect of Wales’ approach to search. “It’s not a good thing that we are getting search results from a handful of very large players and we have no idea how they are generated,” he said when he announced the purchase at the O’Reilly Open Source Convention in Portland, Oregon. “To some extent this is a political thing…Search is part of the fundamental structure of the Internet and should be transparent and open.”
Working on the software isn’t the only way in which volunteers can help the Search Wikia project succeed. The software is designed to be distributed on lots of computers. Much as SETI@Home or Folding@Home works, participants download the software at http://www.grub.org/, and when they are not using their machines, Grub uses their spare CPU cycles to index the web.
Humans will also be directly involved in the results to provide assistance with editing. Human editors could be used to handle tasks that computers still do poorly, like distinguishing when “bass” refers to a fish or a guitar, or “apple” refers to a fruit or the company, or even when “palm” refers to a location like Palm Beach, a hand, a type of tree, or a personal digital assistant.
Another open source element of the Wikia search engine will be Lucene. Lucene is “a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform,” according to its web site. The latest version is 2.2, released in mid-June.
Next: Issues >>
More Search Engine News Articles
More By Terri Wells
