A Few Words About Spiders

As a website owner and someone concerned about Search Engine Optimization, you don’t have to suffer from Arachnophobia. Spiders are your friends!The search engines deploy programs called spiders to transverse the web and index all content available on the internet. These programs follow a set of rules defined by each search engines algorithm. Spiders are in search of information, and SEO is the process of providing information to these spiders in a manner that will boost relevancy.Spiders follow links on your website to other pages within your website, and to other websites to which your website links. Factors that influence spiders, and crawlabilty:Robots.txt file – most robots will look at this to determine where they can and cannot go on the website. In some cases if a spider cannot find this file it will not index the website.Of course, don’t forget about a Google sitemap. (www.xml-sitemaps.com) is a good starting point. A Google sitemap doesn’t guarantee inclusion by Google, but it most certainly doesn’t hurt. It enables Google to properly and easily index your site. It’s also a good idea to have a separate XML site map and an HTML version for your visitors to see and utilize. A good example of and XML sitemap is http://www.kaboodleventures.com/sitemap.xml .• URL’s – Clean urls make it easy for the spider to index content. Variables and long url strings will hinder the spiders, if you site is built in ASP or PHP you may need to do a ModRewrite to create clean urls on the fly while still enabling your Content Management System features to be utilizedo Good: http://www.somedomain.com/content/subcontent/pagename.htmlo Bad: http://www.somedomain.com/index.html?catid=8

Comments? Questions? Email Here

© HowtoAdvice.com

How to Advice .com