Google`s Quality Rater Guidelines Leaked - Inside the Raters HUB

(Page 4 of 4 )

• Watch a leaked video that shows a raters control panel

There's an interesting post that claims Google pays $20 an hour to university students to review search results: http://www.searchbistro.com/index.php?/archives/19-Google-Secret-Lab,-Prelude.html

There's also an official Quality Rater job posting on the Google board: http://groups.google.com/group/ba.jobs/browse_thread/thread/bc4f1dec633b9120/84689e434c38a1fb?q=%22search+quality%22+google+jobs&rnum=3&hl=en&pli=1

A search for EWOQ gives the official EWOQ link, which cannot be accessed without authorization.

There are around 15 pages in the document devoted to guiding new Raters inside Google's Raters Hub.

Raters Workflow:

• Acquiring Tasks

• Starting to rate

• Submitting your initial rating

• Re-rating unresolved Tasks

• Commenting

And here's an interesting image of a Raters control panel:

Here's where Googlers rate content:

In the control panel, raters can select between "Live Page" and "Cached Page." If the live page doesn't work, but the cached page does, raters have to rate the cached version. If both links do not work, the page is assigned to the "Didn't load" category.

Moderators resolve "unresolved tasks" or tasks that raters refuse to rate. Before flagging a task as unresolved, raters must provide comments.

Here's how comments look:

And here's the lingo Google raters use in comments:

-V (Vital) - DL (Didn't Load)

- Usf (Useful) - FL (Foreign Language)

- Rel (Relevant) - MAL (Malicious)

- NR (Not Relevant) - PPC (pay-per-click)

- OT (Off Topic)

Google's Human Secret Weapon

In the next article we'll cover how Google raters identify and treat spam, and what is considered spam.

Danny Sullivan in "Search 4.0" stated that search engines are putting humans back in search via human rater programs such as the one discussed in this article, and behavioral data collected from users.

I believe it makes common sense to use human raters to filter out search results, but the biggest question is: to what extent? Aaron Wall of SEO Book claims that Google has close to 10,000 quality raters. How true is this? I have huge doubts, but when the two best known SEO figures speak of large human influence in search, you'd better listen. Danny hangs out with Eric, Sergei and Larry, while Aaron has engineer friends on the inside, so they ought to know what they're talking about.

As for the guidelines, make you content the best you can and you won't have to worry about raters. In fact, they'll prop you up.

DISCLAIMER: The content provided in this article is not warranted or guaranteed by Developer Shed, Inc. The content provided is intended for entertainment and/or educational purposes in order to introduce to the reader key ideas, concepts, and/or product reviews. As such it is incumbent upon the reader to employ real-world tactics for security and implementation of best practices. We are not liable for any negative consequences that may result from implementing any information covered in our articles or tutorials. If this is a hardware review, it is not recommended to open and/or modify your hardware.

Comments On: Google`s Quality Rater Guidelines Leaked

· Good article. If Google even had 20,000 reviewers, would that be enough to make a...   >>> Post your comment now!