Duplicate Content Tips: Google on Cross Domain Content Duplication
| Thursday, December 24, 2009
||Last week Google put out a post through their Google Webmaster Central blog on how to handle legitimate cross-domain content duplication. (Thanks Charlotte for pointing this article out to me). The article is a great piece on ways of handling duplicate content across different websites, something that the canonical tag did not address when it was released in February of this year.
The Google post announces support of the cross-domain rel="canonical" link element. In addition, the post mentions four ways to handle cross-domain duplication:
There were some great questions addressed in the post as well.
- Choose your preferred domain - this can be done through Google Webmaster Tools to help identify an authority. Remember that when confronted with duplicate content, search engines will generally take one version and filter the others out.
- Reduce in-site duplication - through the whole canonicalization process as illustrated by Google. Within Google's statement, notice the part about how link popularity is still a key factor with their ranking algorithm? To gain more control over how your URLs appear in search results, and to consolidate properties, such as link popularity, we recommend that you pick a canonical (preferred) URL as the preferred version of the page.
- Help the Search Engines find your content - According to Google:
Enable crawling and use 301 (permanent) redirects where possible
Where possible, the most important step is often to use appropriate 301 redirects. These redirects send visitors and search engine crawlers to your preferred domain and make it very clear which URL should be indexed. This is generally the preferred method as it gives clear guidance to everyone who accesses the content. Keep in mind that in order for search engine crawlers to discover these redirects, none of the URLs in the redirect chain can be disallowed via a robots.txt file. Don't forget to handle your www / non-www preference with appropriate redirects and in Webmaster Tools.
- Through the new cross-domain rel="canonical" link element - more about the canonical link element here.
Q: Do the pages have to be identical?
A: No, but they should be similar. Slight differences are fine.
Q: For technical reasons I can't include a 1:1 mapping for the URLs on my sites. Can I just point the rel="canonical" at the homepage of my preferred site?
A: No; this could result in problems. A mapping from old URL to new URL for each URL on the old site is the best way to use rel="canonical".
Q: I'm offering my content / product descriptions for syndication. Do my publishers need to use rel="canonical"?
A: We leave this up to you and your publishers. If the content is similar enough, it might make sense to use rel="canonical", if both parties agree.
Q: My server can't do a 301 (permanent) redirect. Can I use rel="canonical" to move my site?
A: If it's at all possible, you should work with your webhost or web server to do a 301 redirect. Keep in mind that we treat rel="canonical" as a hint, and other search engines may handle it differently. But if a 301 redirect is impossible for some reason, then a rel="canonical" may work for you. For more information, see our guidelines on moving your site.
Q: Should I use a noindex robots meta tag on pages with a rel="canonical" link element?
A: No, since those pages would not be equivalent with regards to indexing - one would be allowed while the other would be blocked. Additionally, it's important that these pages are not disallowed from crawling through a robots.txt file, otherwise search engine crawlers will not be able to discover the rel="canonical" link element.
Great stuff as always from the Google Webmaster Central team.
Labels: canonical tag, duplicate content
|posted by Jody @ Thursday, December 24, 2009