Search Engines Adding Twitter Content to Their Results

Who would think that a service as apparently inane as Twitter could come out of nowhere to suddenly start making money — and even affecting search results? It sounds strange, but it’s true. Keep reading for a closer look at the trend, and why search engines are starting to include these 140-character bursts in their results.

It’s hard to pinpoint an exact time when Twitter began gaining momentum. The microblogging service, which is now four years old, was just a blip on the radar a few years ago, well after it had been launched. At first, some critics dismissed it as a silly fad, but it began to pick up steam when celebrities began using the service to “tweet” their whereabouts, vent their frustrations, and start the occasional Twitter fight with other random celebrities — in 140 characters or less, of course.

Recently, the San Antonio, TX-based research firm Pear Analytics analyzed 2,000 tweets over a two-week period for six hours each day. Based on their research, the firm was able to break down each tweet on the site into six categories. For example, Pear Analytics found that 41 percent of tweets are pointless babble, while 38 percent are conversational. The rest of the numbers are equally unimpressive, with just 9 percent being characterized as “pass-along value,” 6 percent as self-promotion, and equal parts spam and news at 4 percent each.

When broken down in such a way, one would think that Twitter has little value. After all, how can pointless babble be turned into big bucks? Well, it has. The numbers aren’t clear, but it’s been reported that Twitter has raked in millions of dollars from investors, with the numbers skyrocketing each quarter.

Twitter has literally become a powerhouse; it’s considered one of the top three most popular websites in the world and its creator, Jack Dorsey, is now considered a tech force to be reckoned with. Recently, there have been certain goings-on that would give the service an even higher profile — if that’s even possible. Let’s find out what kind of deal Dorsey, the brainchild behind all things Twitter, has struck and how it may affect your search results in the future.

Twitter’s Firehose

A lot of data flies through Twitter, and that data can be incredibly important to news sites and search engines, among other services. Until very recently, Twitter had limits on the amount of data that could be taken from their streaming API — or firehose — and those limits applied to the estimated 50,000 applications that are believed to be using this data to their benefit.

As we’ll soon find out, Google and Microsoft’s new Bing search engine were some of the first to have unlimited access to Twitter’s firehose of data, which culls information from the service’s more than 50 million daily tweets.

This agreement — or contract — that Dorsey and Twitter have entered into with these search engines puts him in a very good position. It also helps the search engines, as they’re now able to benefit from live information updates in a way never experienced before. Essentially, Twitter has agreed to give specific search engines unlimited access to their data, meaning that any time Internet users look for information using a search engine, relevant results may include tweets. It seems like a simple concept, but it’s changing the web.

{mospagebreak title=Google and Microsoft Lead the Way}

During the middle of 2009, both Google and Microsoft announced plans to integrate Twitter content at the Web 2.0 conference, and though they’ve been doing it for a few months now, both companies are still trying to figure out how to display the information in relevant search results and how to choose which information to display. Figuring out how to do both successfully is proving to be more difficult than previously thought.

Though these two companies have led the way together (and Yahoo is following in their footsteps, as we’ll soon find out), both search engines are battling it out to see who can best make use of Twitter’s firehose by analyzing, ranking, and displaying the real-time data in a relevant way. As mentioned, both of these search engines, as well as Yahoo, have recently acquired the rights to Twitter’s API stream, but they’re also trying to figure out how to integrate real-time data from similar services. All of this integration is in its earliest stages, but it’s been made very obvious that this will be the next major fight in the search market.

It should also be pointed out that Yahoo seems to have cut a better deal with Twitter, but Google still remains by far the most popular search engine, and before long, users will be able to find anything that anyone puts online within seconds using Google. As we’ve said, though, we’ll have to wait and see if any of the information proves to be useful or helpful in any way.

It only seemed apropos that a few months ago Google presented its vision for real-time search to the media while in front of the Computer History Museum, at the time claiming that they were well on their way to making history as well. This was when Google first announced it would feature real-time results via Twitter and other services, and since the end of last year, users must have surely noticed a new box featured on the search engine entitled “latest results.”

This new box appears on the main page and enables Internet users to scroll through up-to-the-minute real-time results associated with their query culled from sites such as Twitter, FriendFeed, and Google News. Not only that, but it also includes brand new web pages as they’re being created, which is actually quite impressive.

Though Microsoft’s Bing search engine has already jumped on this hot, new real-time trend, it doesn’t display its data as Google does. Though Bing does display Twitter updates and pertinent search results from various blogs, those results actually do not get integrated with the main page.

According to the tech giant, this is just the beginning of their plans. Google intends to display many different kinds of content culled from the Internet in their “latest news” box. This is exactly how Twitter is becoming more popular and lucrative, and companies like Google are benefiting from their data. Google paid Twitter an undisclosed amount because Google, as it’s always been, is very concerned with the speed at which the latest information gets to its users. Now that the two companies have inked this new deal, Google’s search engine will save a lot of time by now being able to harness all of the data from Twitter’s firehose, as opposed to having to pull it out of the Internet itself.

{mospagebreak title=Yahoo Jumps on Board}

As we’ve alluded to several times, Yahoo is the latest — and third — major search engine to cut a deal with Twitter, which now allows the company unlimited access to their firehose of data. Like Google, Yahoo will integrate Twitter content into a box on its main page just above the most popular recent searches, like “Obama” or “2010 Olympics,” for example. Unlike Google, however, Yahoo will be able to integrate even more data, because their purchase of access to Twitter’s firehose also enables them to include real-time Twitter content not just to search results, but to Yahoo profiles as well.

Yahoo is actually integrating the real-time content they’re receiving in a number of inventive ways that other search engines, such as Google and Bing, haven’t managed to do yet. For example, all Yahoo users will not only be able to view their Twitter feeds and post tweets to the service directly from their Yahoo accounts, but Yahoo also intends to extend the same idea to Facebook users by enabling users of the popular social networking site to update their profiles from their Yahoo accounts.

According to reports from CNET, it may take Yahoo longer than Bing and Google to integrate their own information with Twitter’s data so that the two streams of data from the two separate sites are running smoothly and seamlessly together, but once that small hurdle is passed, Yahoo will be offering some of the most information-laden, heavily integrated services currently available on the web — and these days, that’s saying something.

According to Yahoo, their desire to integrate popular websites like Facebook and Twitter into their already-existing services is two-pronged: it’s beneficial to their business and it helps their users. This type of integration makes staying in touch and managing their multiple social networking accounts much easier for the average user, but it also greatly shortens the amount of time these same users are spending on competing sites, like Facebook.

According to Jim Stoneham, Yahoo’s vice president of communities, “This is a way to help users integrate their social-networking lives into their Yahoo e-mail and instant-messaging lives,” he said. "It’s a continuation of our drive to make Yahoo the one place to be.”

Like Google’s deal with Twitter, the financial terms of Yahoo’s deal with the microblogging service have not been released. That being said, Yahoo has said that they negotiated their own deal with Twitter, as opposed to allowing Microsoft to integrate real-time results under their search partnership. Yahoo said this was done “because content deals signed by either party before regulators approved the search deal would not automatically transfer over.” Also — as we already mentioned — Yahoo’s deal is a bit different from Microsoft’s in the way that it enables Twitter users to link their Twitter accounts to their Yahoo accounts.

This is a very interesting time in history, as it’s the first time the average person literally has all of the information in the world at their fingertips and now, it’s being offered in real-time, faster than ever. What will they think of next?

[gp-comments width="770" linklove="off" ]