The Times and Rupert commit search engine suicide

Forget SEO, News International is covering new ground in SES (Search Engine Suicide). They announced in March that in June 2010 a paywall on all content would be put in place. That happened earlier today.

302 Redirect

For a while earlier today, today’s paywall launch was search engine suicide due to the way that tens of thousands of timesonline articles indexed by search engines were being redirected.

Pretty much every URL for a number of hours today was being redirected to an index-page on a different domain, which then also displayed a paywall notification layer.

Here’s an example:

There you can see the “302 Moved Temporarily” redirect. This tells Google and other search engines is to keep checking this URL as at some point it will start to serve content of its own again. Google generally honours this type of redirect and uses the content of the destination page for indexing and ranking purposes.

What we can spot here though is that The Times article we tried to read (“Fears that vuvuzela horns could harm World Cup football fans hearing” from June 9th 2010) is being redirected to a different top level domain http://www.thetimes.co.uk and this is the case for every redirect I’ve checked. Tens of thousands of articles are being redirected to the same URL. This is starting to look to Google very much like the old hijacking trick by spammers.

301 Redirect

But it doesn’t end there. Once we’ve followed the first redirect, we then get hit by a second one. This time a “301 Moved Permanently” redirect status code.

This tells us (and the search engines) that http://www.thetimes.co.uk/ has been moved forever and should not be shown in indexes any more and any “value” (Pagerank) the old page has should be transferred to the URL pointed to by the redirect, which is http://www.thetimes.co.uk/tto/news/

Pop goes your index

So what does this mean? Well, Google will receive same page for all articles, plus that content is being served from a different domain to what it started with. The likely hood is that the entire site will be dropped from the index as punishment for spamming tactics. Best case is that the 60,000 plus articles currently indexed by Google will be replaced by http://www.thetimes.co.uk/tto/news/

Update: 20:31

Whilst I’ve been writing this post, the redirect behaviour has changed a number of times. It seems to be stabilising and reflecting what The Times outlined in their announcement. Articles from timesonline.co.uk that are currently indexed by the search engines are still available, for free, and don’t redirect anywhere.

New articles published after the launch of the paywall are being 302 redirected to the timesplus.co,uk start page with a paywall “login” splash.

This means that no new Times article content will be indexed by Google; it will just receive very similar content for every URL. We are going to see News and opinion from The Times, sign up now for an exclusive preview of the new Times website” or something similar a lot for Times pages in search results from now on.

Update: 16 June 2010

If the erection of the paywall wasn’t crazy enough, I’ve also spotted that The Times have stopped updating all of their timesonline.co.uk RSS feeds

Why on earth would you stop feeding article teasers to thousands of loyal readers who have taken the time to subscribe to your news feed? Surely these are exactly the kind of people who might actually pay a few pounds to get through the paywall!