Perhaps that’s a slightly extreme comparison. After all, Google, Yahoo, and the other search engines are supposed to serve as our reference librarians to finding all the material we need, right? Well, according to research funded by the Economic and Social Research Council (ESRC), they’re not doing as good a job of it as many of us would like. At the very least, the researchers think they could be doing a better job when it comes to finding key web sites covering the sciences.
The researchers, all from the Oxford Internet Institute, were studying how the Internet is changing the way people look for authoritative science information. In particular, it focused on academic researchers studying topics such as HIV/AIDS, climate change, terrorism, and the Internet and society. The researchers doing the study discovered that many of the organizations considered to be of high importance to the academics studying those fields didn’t show up in the top 30 results in Google.
Let’s look at the first example, HIV/AIDS. Academic researchers studying that field used certain “national journals, charity organizations, statistics and public sector organizations” for source material, according to the press release from ESRC. However, none of those specific items appeared in the top 30 results in Google for generic key words. Now you can argue that someone who is an academic researcher probably wouldn’t be using generic key words, but that may not be entirely fair. If the information is on topic, why shouldn’t the sites turn up respectably near the top for a generic key word search?
One possible answer is that at least some of those who run scientific web sites don’t have a clear conception of SEO, or the need for it. Here’s a point from the ESRC press release that I think most SEOs will find to be intuitively true: “A fundamental observation was that, despite popular perception, the Web is far from being a neutral source of information. It has a particular structure that steers the search in directions that may not be intended by the user and so makes some sites more accessible than others. Search engines such as Google play an increasingly important gate-keeping role that will influence the information that is found. They can shape ‘winners and losers’ by means that are not always apparent and moreover do so in a manner which can vary according to subject matter.”
You may or may not be in charge of a scientific web site. But either way, you face the same issue: how do you make your site more visible? Well, one way to do it is to help your visitors find whatever they’re looking for. Despite appearances, the search engines do not have a total lock on that little trick; indeed, some of them make very nice tools that can assist you in this task.
Take Yahoo, for example. In February it released something that looks truly original; sadly, they weren’t exactly trumpeting it from the rooftops, so I only heard about it recently. It’s called Yahoo Pipes, and if you know anything about the pipe (|) command in Unix, you already understand conceptually what it does. I don’t have the room to do a full review in this article, but I may do one later if there is interest.
Yahoo’s overview says that Pipes “is a free online service that lets you remix popular feed types and create data mashups using a visual editor. You can use Pipes to run your own web projects, or publish and share your own web services without ever having to write a line of code.” What does this mean exactly? Basically, you can take a number of web feeds, RSS feeds, or other sources of online information, run them through several programs in order (Pipes calls them “modules”) and get the results – yes, just like the pipe command in Unix lets you take one program, run it, then put the results into another program and run it.
At this time of writing, Yahoo offers almost 30 Pipes modules that can be strung together in various orders. Users string the pre-configured modules together onto a canvas, “wiring” them together in the Pipes editor. It’s a very visual, drag-and-drop interface. It’s even pretty, which probably goes a long way toward making it fun to use (and easier for users to stick with, playing around until they have a Pipe that does exactly what they want).
By themselves, Pipes can be great time savers for aggregating only the information you want. But Pipes come with social options. Sure, they’re on Yahoo’s server, and you call them as you would any other feed. But you can also publish them and share them with the world. Visitors to Yahoo’s Pipes site can browse Pipes; each one is listed with its name, author, date of publication, and how many times it has been run.
The list is well worth browsing; Pipe authors have been very creative. It’s not just news they aggregate; how about eBay items in a particular price range, or apartments near schools for those with families? Think about your web site, and you’ll probably be able to create a Pipe with the needs of your visitors in mind.
I reviewed Google’s Custom Search Engine (CSE) when it came out in November of 2006. Google’s approach to helping users build exactly what they want to help them find the information they want is no better or worse than Yahoo Pipes; it’s just different. Since I already reviewed it, I’m not going to go over it completely here, but I will touch on the high points.
The idea behind Google CSE is to let users build a search engine that is focused specifically on their own interests. It can be built by one person working alone, several people, or even a large group. As with Yahoo Pipes, you don’t need to be a programmer to make it work.
Creating a CSE is a simple four-step process:
- Place a search box on your web site.
- Specify the sites you want to include in searches.
- Customize the look and feel to match your site.
- Invite your community to contribute to the search engine.
You specify key words that you want your search engine to focus on in its searches; when I did the review there was a character limit on the text box for the key words, so you want to be careful about the words you choose. Then you choose the URLs which your engine will search either exclusively or predominantly. Here you’re not limited so much; you can even add URLs later (that’s part of building the search engine, after all). Better still, you can specify that your search engine either include OR exclude the site, or even just that one page. So if, for example, you’re building a CSE that covers Jewish history, you can choose to exclude web sites which distort that history.
The applications for Google CSE are obvious, and almost too numerous to mention. I mentioned them in the review, of course. Obviously, a CSE on your web site is major linkbait, but there’s another reason to build it, or a Pipe, or some other related item: it will help your visitors find the information they need.
Of course you want your site to be visible, and you’re going to do whatever you can as far as search engine optimization to make that happen. That includes regularly adding fresh content to your site. Ironically, Michael Kenward, commenting on the ESRC-funded research, noted that scientists don’t seem to understand that point. “Our take is that scientists need to be a bit more careful in how they manage their web sites. Not only do they have to make them user friendly, they should also ensure that they stay up to date.”
Much as you might want to, though, you probably won’t be able to provide your visitors with everything they may want or need that’s related to your field right on your web site. That’s where things like Yahoo Pipes, Google CSE, and certain forms of linkbait come in. Even if some of these technologies take a web surfer off your site temporarily, they deliver what your visitors are really looking for – and any web surfer that finds what he or she wants at a particular site is likely to bookmark it and come back.
With the Internet more like a vast and varied ocean than an information superhighway – or a huge library with all the books scattered on the floor – these kinds of tools and technologies can serve an important purpose. Sure, they’ll make you and your site more visible; but they’ll also encourage your visitors to see you as a helpful guide and an authority in your area. That kind of positive PR is worth its weight in gold.