The Art and Science of Link Spam Detection

Posted by Eric Ward on January 24, 2011 to

The past couple weeks have seen several high profile online publications discuss the quality of Google's search results. The common thread has been that Google's search results are getting worse over time, and especially so during the past year or so. The noise got loud enough that Matt Cutts responded on the Official Google Blog, and I was glad to see him tackle the questions head on. SearchEngineLand has a nice recap if you missed it.

I agree with Matt's assessment that Google's results are actually better today than they have ever been. I also agree that there's more marginal and "shallow" content today than there has ever been. The link spam of yesterday has been replaced by "content farms", most of which are the same link spam and spammers, driving nicer cars and wearing nicer clothes. I know it's more complex than that, but the bottom line is fake is fake. Spotting a fake is where the challenge is and has always been. For Google, for Link Insight, for you and me.

One quote from Matt really caught my eye. He wrote...

"As we’ve increased both our size and freshness in recent months, we’ve naturally indexed a lot of good content and some spam as well. To respond to that challenge, we recently launched a redesigned document-level classifier that makes it harder for spammy on-page content to rank highly. The new classifier is better at detecting spam on individual web pages, e.g., repeated spammy words—the sort of phrases you tend to see in junky, automated, self-promoting blog comments".

His quote really drives home for me that we are on the right track with Link Insight. Why? Because Link Insight can use a document level classifier, for the same reason Google does. To help detect spam at the page specific level. For link building, detecting spam helps you steer clear of link seeking targets that wont help you. You don't waste time and money chasing the wrong rabbit. Put more simply, if a page is deemed to be spam by Google, or Link Insight, then links from that same page are not going to provide positive trust signals, so why pursue them?

There's much more to the process of spam identification, and as spammers have become more sophisticated, so too have the spam cops and bots. It's a game of leap frog, really, with the engines and link building tools constantly trying to stay ahead of each other, learn from each other, outsmart each other.

Whether or not Google's results are better or worse is a matter of opinion and open to debate, and I'm a Google loyalist at heart. They provide the best search experience for me. Yes, there's still junk in there. Yes, spammers still find ways in. But at the same time, I don't envy any website depending on trickery for their success. Fake is fake. It's just a matter of when you get caught, and the clock is ticking. Here's one last quote from Matt.

"One misconception that we’ve seen in the last few weeks is the idea that Google doesn’t take as strong action on spammy content in our index if those sites are serving Google ads. To be crystal clear: Google absolutely takes action on sites that violate our quality guidelines regardless of whether they have ads powered by Google"

Tick tock...