You see the same mistake virtually everywhere on the Internet. Huge companies like Apple and even SEO firms commit this error, so it’s not surprising average websites also suffer this affliction. The problem is that there are multiple URL addresses for each page.
Search engines see the results as different sites. Search ranking strength is diminished or worse penalized as duplicate content. The technical term is URL normalization. Unfortunately, much is left to interpretation.
No matter how someone types a URL, it should be transformed to have https, www, and end with /.
HTTPS Recommended
Since Google announced HTTS as a ranking signal six months ago, many organizations have half heartedly implemented SSL certificates in response. Unfortunately, very few have made their sites SSL by default:
http://www.example.com/
https://www.example.com/
Customers wonder whether the site is truly secure and search engines see two different results. SSL by default means all results will be transformed to beginning with https. Does your website show customers you are serious about security?
WWW Recommended
The next quandary people stumble on is the decision of a naked domain versus one beginning with www:
http://example.com/
http://www.example.com/
While the naked domain looks cleaner, the www prefix is recommended. Use naked domains for marketing and less typing, but don’t allow your website to return both addresses.
Remove Directory Index
Often websites also erroneously return a start page, as well as the standard web address:
http://www.example.com/index.htm
http://www.example.com/
Home pages should not be linked internally or returned in search results.
Friendly URLs Recommended
Although not a part of normalization, modern sites have clean URLs with no file extensions:
http://www.example.com/about.htm
http://www.example.com/about/
Even legacy sites should only return friendly URLs that are easy to remember.
Trailing / Recommended
Most sites also return address with and without a trailing slash:
http://www.example.com/about
http://www.example.com/about/
Technically, a trailing slash denotes a directory, while no slash is a file. However, since search engines see two different results, all addresses should end with a trailing slash.
URL Rewrite Fix
There are many web development platforms, but we tend to see ASP.NET more often which utilizes URL rewrite rules in web.config:
<system.webServer>
<rewrite>
<rules>
<rule name=”Convert to lower case” stopProcessing=”true”>
url=”.*[A-Z].*” ignoreCase=”false” />
<action type=”Redirect” url=”{ToLower:{R:0}}” redirectType=”Permanent” />
</rule>
<rule name=”Redirect htm to friendly” stopProcessing=”true”>
url=”(.*)\.htm” />
<action type=”Redirect” url=”http://www.example.com/{R:1}/” redirectType=”Permanent” />
</rule>
<rule name=”Redirect to www” stopProcessing=”true”>
url=”.*” />
<conditions>
<add input=”{HTTP_HOST}” pattern=”^example.com$”/>
</conditions>
<action type=”Redirect” url=”http://www.example.com/{R:0}” redirectType=”Permanent” />
</rule>
<rule name=”Redirect to HTTPS” stopProcessing=”true”>
url=”(.*)” />
<conditions>
<add input=”{HTTPS}” pattern=”^OFF$” />
</conditions>
<action type=”Redirect” url=”https://{HTTP_HOST}/{R:1}” redirectType=”Permanent” />
</rule>
<rule name=”AddTrailingSlashRule” stopProcessing=”true”>
url=”(.*[^/])$” />
<conditions>
<add input=”{REQUEST_FILENAME}” matchType=”IsDirectory” negate=”true” />
<add input=”{REQUEST_FILENAME}” matchType=”IsFile” negate=”true” />
<add input=”{URL}” pattern=”css$” negate=”true” />
<add input=”{URL}” pattern=”jquery$” negate=”true” />
<add input=”{URL}” pattern=”bootstrap$” negate=”true” />
<add input=”{URL}” pattern=”modernizr$” negate=”true” />
</conditions>
<action type=”Redirect” url=”{R:1}/” redirectType=”Permanent” />
</rule>
</rules>
</rewrite>
</system.webServer>
Test Your Website
There are dozens of things you can do improve the ranking of your website in search. However, having a consistent format for URLs will help all of your endeavors.