Duplicate content is one of the most common issues that can plague websites from not being found in search engines, and most wouldn’t even know it was happening.
There are a few types of duplicate content. The problem with each of these scenarios is that either a) the search engine can not figure out which page to rank (they do NOT want multiple sites on the first page of results linking to the exact same content), so they usually rank none of them; or b) the search engines think you are spamming them. Either way, you don’t want to have these plaguing your results, and they are fixable.
The first is the lesser of the infractions. Most websites will have a “www” version, and a non-www version. Technically, the search engines see this as two different websites; i.e. www.yoursite.com and yoursite.com. The version that is your preferred domain is called the “canonical URL”, and the other version should redirect to the preferred version using a 301 permanent redirect. You can identify the canonical URL to the search engine by using a “rel=canonical” tag in the header of the primary page.
Some businesses go with the strategy of having multiple domain names to compete in the search results. For example, they may have “www.redshoes.com” and “www.maroonshoes.com”, which is fine – except that both sites are exactly the same. Word for word, page for page, the content is exactly the same. Instead, choose the domain with the most authority (site age, higher Google PageRank, more inbound links, etc.) and redirect the lower authority one to the higher authority one.
Other companies with locations around the world will have their website replicated on multiple country extensions, such as .com, .co.uk, and a .ca. This can be a good strategy, but do NOT copy the content verbatim from one site to the other. To do this properly you will need to first repurpose the content to fit that country (such as using “colour” on a UK site), as well aaa getting links from sites or directories that originate in that country. Hosting the site in that country will help too. Doing all of this will hep you rank higher locally in that region.
Update 6/29/10 – new information suggests that Google is NOT penalizing corporate sites that have similar, or perhaps even the same content on multiple country-specific domains. The search engines realize these are not intentionally duplicate content sites; however, the country version should still have language appropriate for that region of the world.
Sometimes we see websites with multiple instances of the homepage for affiliate marketing or tracking purposes. They may have something like:
All of these pages are exactly the same, except only one of them is the preferred; the canonical URL. All others should have a NOINDEX meta tag to prevent the search engine from indexing the page and then getting confused later.
Think you have a duplicate content problem? Run your website through the free SiteJuice 2.0 page analysis tool.