Manber came to Google in 2006 with an impressive resume. His past positions include stints as a computer science professor at the University of Arizona, a senior vice president at Amazon and Yahoo’s chief scientist. In the 15 or so years he has worked on search, he has seen attitudes toward it change tremendously. As he explained, “When I started in academia and I said I’m working on search, they looked at me and said, ‘What do you mean you’re working on search? Did you lose something?’”
Back in the early 1990s, the search field was divided by topic — chemical search, legal search, medical search, and so on. And only professionals did search; you would tell someone what you wanted to find, and they would find it for you. Some of the earliest “search engines” were more like directories of information rather than something into which you could put a term and hope to get an answer that gave you what you were looking for.
As the Internet grew, and search engines improved, that changed. Manber understood that the information revolution touted in the 1990s meant that “it’s not enough to store the information and move it around, you have to find it…The ability to find things among huge amounts of information is the key factor.”
Back in the early days of search, however, this need wasn’t obvious to everyone — and even today, it isn’t as common as you might think. Manber says that one of the perks of his position at Google is that “I don’t have to tell anybody around here that search is important, and that’s a very nice luxury to have.”
In interviews, Udi Manber noted several interesting points about the development of search. When some of the first search engines came out, they may have been very primitive by our current standards, but they looked pretty mature at the time. “My best analogy is that a 15-year-old thinks he’s very mature. A 19-year-old thinks he’s extremely mature,” Manber observed. But we still experienced certain frustrations. “Ten years ago, if you actually found an answer to some specific question, it was, ‘Hey, look at this, it’s so cool!’ It was an event,” he noted.
Now, of course, we expect to find what we’re looking for online. When we don’t find it, we assume that something is wrong. It’s a major challenge for Manber and his team, but that challenge has stimulated tremendous innovation in the field. He likens it to seeing science fiction become real every five years. “When the first search engine appeared in ‘94, compared with when I came out of academia in ‘99, compared with the way it was in 2003, compared with the way it is today — every five years there have been just incredible advances. What we do now, we couldn’t have foreseen 10 years ago…People expect more from us,” he said.
In all that time, though, the goal of search engine scientists remains simple: give the searcher what they’re looking for. Even for Manber — perhaps especially for Manber — when he doesn’t find exactly what he needs when he does a search, it’s frustrating. And he and the other search engineers at Google seem to take it personally.
“When someone finds an example of something that doesn’t work that should work, we think, ‘How can that happen? How is it that we missed something?’” Manber explained. The cause could be anything from a weakness in the algorithm to something missing from the Internet itself, such as when a restaurant or other brick-and-mortar business leaves something important off of their web page.
Before explaining how Google goes about improving searches, and the areas into which they’re looking, Manber offered some excellent advice to users who want to improve the odds of finding what they need, and web sites hoping to improve their odds of being found. “The content provider should think about how users will look for their content, and the user should think about what words people use to write about their content.” If you’re searching for something, think about what you would expect to see in the actual page; don’t treat the search engine like a person to whom you are trying to explain what you need.
Even though Manber advises searchers not to think of search engines as people who might understand their queries, he notes that Google is working in that direction. “We will take your query and try to ‘understand’ it and match it as best we can to the content we find on the web,” he explained.
As you would expect, this takes a lot of work and adaptation; the search team at Google made no fewer than 450 modifications to the algorithm last year. That’s more than one every day — and may go some way to explaining why it’s sometimes best to wait a few days when you see your position in the SERPs change rather than immediately assume that you’ve done something wrong and must fix it right away.
Likewise, you should not assume, when you go through that kind of bounce, that Google has it in for you personally. Yes, there are people working on the algorithm at Google, but they are not personally manipulating the results. Contrary to a recent Ranked Hard SEO comic strip, Manber said that “At Google we do not manually change results…we do not have the capability…We made that decision not to put that capability in the algorithm — we have to go and actually change the algorithm. That is, we have to find what weakness in the algorithm caused [a poor result] and find a general solution to that, evaluate whether a general solution works and if it’s better, and then launch a general solution.”
This slows down the process, but it also reduces bias and forces more discipline on the search engineers. Of course, not everyone believes that the algorithm can’t be manipulated manually, and inevitably someone will tell a story that seems to “prove” that Google can reach in and tweak things by hand. One of the comments to a Popular Mechanics interview of Udi Manber gave the example of posting “a comment about a funny result and within 15 minutes of it being posted it was gone. The result was a Wiki article ranking #2 for the search GOOGLE. I commented on it at a place where I know Google engineers read and it was gone fast.”
As an answer to that, it’s worth noting that changes in Google’s algorithm and search results do propagate fast. “If something new happens in the world and you search for it,” Manber explained, “I’m not going to give you an exact time, but within an hour you will see in the direct results pages that relate to that story.” And items quickly make it from the index to the search page. “It’s also the case that if you do the same search on different days you may get different results, because some of the results are things we indexed five minutes ago.”
It’s also worth noting, in reference again to the Ranked Hard comic strip, that Google maintains a “church and state” separation between the advertising side and the search engine algorithm. The search engineers hold weekly meetings where they look at the proposed improvements “and we look at all the evaluations and we make decisions — revenues and any effects on ads do not come into those meetings,” Manber emphasized. “We don’t even know what the effects are. We make decisions solely based on how good it is for search.”
Unfortunately, it is Google policy to NOT comment about future plans, so in his recent interviews Manber could not come out and say what he and his team are working on to give us all a better search experience. “There will be lots of rocket-science things that will come along, but those I can’t talk about,“ he explained. Rest assured, though, they’re looking at all the current trends. And they have recently implemented some fascinating options that have not received a lot of press.
For instance, one new feature makes more of the world available to searchers. It’s called CLIR, for Cross Language Information Retrieval. The feature takes a user’s query, translates it into another language, performs the search in that language, and then translates the results into the user’s language. According to Manber, that gives users access to results in about 12 different languages. “So if you’re a user in Egypt, for example, and you only speak Arabic, you can write the query in Arabic, ask to translate it into English. When you then click on the results, it will translate the Web pages to Arabic for you — all of it done by Google translation,” Manber explained. While anyone who has used Babelfish can tell you that automated language translation still leaves something to be desired, Manber believes this feature has a great deal of potential to open up the whole world to everybody.
What about social search and social networking sites? Does Google see this as the wave of the future? Manber notes that Google sees search as getting signals, putting lots of signals together and reading them. It stands to reason that the better the quality of your signals, the better the quality of your results. “Signals from people are the best signals,” Manber observed. “We have several tools — and we’re going to launch many more — that will encourage people to contribute more.” Even so, Manber does not agree with the idea of creating search results manually; one does not follow from the other.
Not surprisingly, Google will continue to work on personalized and universal search, seeing these as ways of giving users more options and a greater chance of finding exactly what they’re looking for on the first search. It will also continue to tune results by location; the country in which you’re searching will affect your search results, and they may not be the same as the same search performed in a different country, even if the searches are performed in the same language. And it will also continue to fight spam and to keep porn out of results where it isn’t appropriate; Manber says it’s harder to fight spam than porn, because porn web sites typically aren’t trying to trick you into thinking they’re something they’re not.
Google is also working on getting better at video and photo search. According to Manber, the problem isn’t quite as bad as not knowing what’s in the picture, since they can tell what it is with text. “The problem is you want to look at the Hearst Building with the sign from the right angle with the sun up above,” he said by way of illustrating that it’s understanding the quality of the image, and the details desired, that’s the real challenge. “That’s going to require some combination of some image processing and some information about it. The metadata around the image is going to get more important.”
But whatever features Google adds in the future, or new adjustments it makes in its algorithm, it all comes back to one thing: does it serve the searcher? Does it help the user find what he or she is looking for more quickly, more efficiently, more easily? And that is why, if Manber’s informal observation holds true, in five years we’ll be able to accomplish the kinds of things with search that we can only imagine in science fiction now. Buckle up; we’re in for an exciting ride.