This is a small file that points and guides spiders as to what they can index and what they should keep away from. There may be areas on a site that that you want to block off from spiders so that that information in those sections does not (or should not) end up on the web. CMS systems have areas that you don’t want found on the Net so you use the robots.txt file to ensure that. WordPress, for example, suggest you block about 13 folders through your robots.txt file, Joomla about 10. We have tagging on the blog portion of the site for pages. Because of this you could reach the same page in a number of different ways and thus a number of url’s. Therefore we have the labelling URL’s blocked so that they can’t be indexed and the page doesn’t end up on the web with two url’s and giving us a duplicate content issue. You have to be careful of the syntax in the file for example: User-agent: * Disallow: / This blocks everything, it instructs robots not to index anything User-agent: * Disallow: Without the trailing slash, this now means index everything. It’s only a…


Page Title and Descriptions

Page titles and descriptions are your “elevator pitch” in the search results and you only have an instant to make a good impression on a potential visitor. Page titles are incredibly important in terms of SEO and the title of a book they inform the potential reader as to the pages contents. Search engines it appears place a good deal of weight in titles and the keywords in them. Simply by manipulating the pages titles can have an immediate and positive effect on your search rank. Actually seomoz, the Search Specialists conduct an annual survey of around 80 of the SEO heavy hitters worldwide to see what they believe are the most important factors in aiding a site to rank. Last year, of all the “on page factors” possible, keywords in a page title was ranked number one. Looking at this part of the process is an exercise in futility unless you have both analytics running on your site and have signed up to a Google Webmaster Tools (GWT) account as they are two great aids in looking at a sites titles and descriptions. If either of these is only being set up then, ideally, some time should have passed…