Google Audio Indexing, or “GAudi” for short, works by taking the audio tracks of YouTube videos and automatically converting them to text so they can be searched. Results from searching for a user’s keyword are ranked based on the spoken content of the video, its metadata, and its freshness. Google’s speech research group developed the speech recognition technology underlying GAudi in house.
If this technology sounds familiar, it’s no wonder. Back in mid-July 2008, Google unveiled the Google Elections Video Search gadget for iGoogle. This tool transcribes and indexes the spoken content on YouTube’s Politician’s channel. It also boasts the same speech recognition technology as GAudi.
At the time of writing, the search giant is concentrating on videos uploaded to YouTube’s Politician channels. Nobody believes that it will continue to limit its audio indexing to that one area, however — especially after today’s election. While it has proven to be vastly useful to concerned voters who want to discover where the various candidates stand on the major issues – to say nothing of the clever pranksters who put together the “Barack Roll” — one can easily envision many more tasks the service could helpfully perform.
I’ll discuss a number of these tasks somewhat later in this article. I’ll also bring up some of the implications for SEO; I don’t doubt that most of you are already thinking ahead in that area. Since this tool is still in beta, I also hope to mention the aspects of GAudi where there is clearly room for improvement. First, however, if you haven’t used the service yourself, I strongly suggest you do so; you’ll need a computer on which you can watch and listen to online videos to get the most out of it. Or better still, just keep reading as I launch into the review section of my article.
At the time of writing, a click on Google Audio Indexing from Google Labs automatically performed a search on the word “economy” and returned the appropriate results. So here’s what my original page looked like:
Okay, this image calls for a little explanation. Actually, it calls for breaking down into the appropriate sections and discussing each one. I’ll even provide close-ups for some of those sections as appropriate.
First, let’s look at the section just below the blue bar at the top. With the links in this section you can alter your search to look for your keyword in videos by all politicians (default), McCain, Obama, or the presidential and vice presidential debates. Searching the debates for “economy” turns up all four debates with more than 10 mentions.
I caught the third presidential debate, which inspired me to do a little search of my own. I discovered that searches need to be done in a particular way. Say you want to search the presidential debates for the phrase “Joe the plumber.” Entering the phrase “Joe the plumber” and then clicking on the “Debates” tab simply took me to the debates videos – and with the “economy” keyword, at that. From that point, changing the keyword to “Joe the plumber” and hitting the “Search videos” button searched all the politician videos for “Joe the plumber,” which was also not what I wanted.
If I’d wanted to do the search from the beginning, I would have had to enter “Joe the plumber,” hit the “Search Videos” button, and then click the link for the debates. This finally turned up the one presidential debate in which “Joe the plumber” was mentioned more often than the economy, though you wouldn’t know it from Google’s blurb (since both were mentioned more than ten times).
A look at the view for this keyword for “all politicians,” however, could easily lead one to think that Google needs to tweak its algorithm, since videos that don’t mention Joe the plumber at all rank above the third presidential debate:
As you can see, the first two videos listed don’t mention Joe the plumber at all, while the third one mentions him only twice. If all four videos really were uploaded and/or indexed four days ago, shouldn’t the one listed last in fact be listed first, since it includes the most mentions of Joe the plumber?
Showing you this picture gives me the opportunity to focus on the left section of a full GAudi page result. You get a link to the video; clicking the link moves the video to the large pane on the right, from where you can play it. The text under the link is pretty self-explanatory; it includes the video’s running time, the number of times the video mentions your keyword, and presumably how long ago the video was uploaded.
Now I want to focus on that big pane on the right.
All of those yellow dots are points in the video where the keyword is mentioned. Just below the video is a search box that lets you search within it. So if you want to see if Barack Obama mentioned health care in this particular speech, you can. In fact, he mentions it once, near the end (and surprisingly, if Google’s technology can be trusted, he never uses the word “change” in this video).
Below the search box for searching within the video you can see a set of clickable buttons with arrows on them. Those take you directly to that particular use of your keyword in the video. Next to the button is a snippet to give you the context of the keyword’s usage, so you can decide whether this is the section to which you want to listen. This is especially handy if you don’t like to sit through an entire speech, or you want to compare the ways various politicians use particular words. Of course, it’s also useful if you want to parody a politician using his or her own words. By default, only the first four uses of the keyword are shown, but you can click a link to show them all.
There is a box below the “show all mentions” link that I had to crop. It includes the URL for the video above it, and links that let you share the video on Digg, Facebook, MySpace, and del.icio.us.
The algorithm GAudi uses delivers interesting results. Enter “homosexuality” and you get few videos; put in “gay,” however, and you get about six pages of them. Putting in “Enola Gay” turns up nothing, however. That’s pretty close to what I would expect.
There is room for improvement with GAudi. It would be nice, for example, if GAudi knew that “homosexual” and “gay” refer to the same thing. Other users of GAudi report stumbling across relatively random results. One noted that when he searched on “Keith Richards,” he got “a speech by Mike Huckabee about pardoning Keith on a reckless driving charge.”
Another improvement I’d suggest was also mentioned by several users on the Google Group for GAudi. If Google has a text version of the spoken content, which one assumes it must in order to be able to do the searches, why can’t the search engine run the text across the bottom of the video as is done for close-captioned TV?
One can only hope that these improvements make it into an expanded version of GAudi. Everyone who has commented on the new service is convinced that Google will expand its use of the technology at least throughout YouTube. The search giant has hinted as much in its list of GAudi frequently asked questions. Under the question “I would like my content to be ‘speech-searchable,’ what do I do?” Google states that “All served videos come from YouTube channels. You should first upload your content to YouTube.”
Assuming Google does expand the tool’s search sphere, what can we expect? Well, one thing we may see is the release of the API, which would allow developers to use it in their own sites in some unique ways. Imagine letting visitors to your web site search every video you have posted for the specific content they want. It could make instructional videos more useful to students trying to review. Converting the entire spoken content of a video – or even just a section of it – to text that can be printed out (which may eventually be possible with this technology) and reviewed at any time could also help someone studying information from a how-to video or online lecture.
Of course, this brings us to the SEO implications. While it is true that Google is currently only searching videos on YouTube, and those only in a particular channel, it is possible that the search giant will turn its probing eyes and ears to other videos. Right now, if you want to be first in line to make your videos more searchable, you need to upload them to YouTube. That probably won’t be true forever. Eventually, it will be every bit as important, if not more, that the keywords you use for ranking your videos actually get spoken in your videos, and don’t appear just in the typical places, such as title, H1, body text of the page on which the video appears, anchor text in links leading to the page, and so on.
Overall, I would say this tool offers a great start to making formerly unsearchable content accessible to users looking for specific things. It can use improvement, but its timing is excellent, both in terms of coming out during the heat of the U.S. Presidential election campaign, and in terms of how popular audio and video content is becoming online. And while you can’t yet discard all of the tricks you use to help the spiders understand what your videos are about, that time may be coming soon.