Facebook Twitter

Design A Spider Friendly Site

Posted on January 18, 2022 by Fred Jensen

To be successful in the search engines it is important to design your web site with the spiders in mind. Using the latest in web page design isn't generally the best thing to do. Spiders do not view web pages like humans do, they need to read the HTML in the page to find out what it is about. Below you'll find ideas about how to best design your web site with search engines in mind.

Don't use frames at all. Some search engines can't spider web pages with frames whatsoever. For the other search engines that can, they can have problems spidering it and sometimes they too can not index the web page. Don't only use image's to link out. You should always use text links to link out to important content on your web site. Spiders can follow image links, but like text links more though.

Use external JavaScript files instead of using Java Script code in the HTML document, using Java Script in the HTML document will make the page size much larger. Using an external Java Script file to do the job will reduce page size and make it easier for both spiders and browsers to download the page. Using Cascading Style Sheets can reduce page size and making the download time much faster in many cases. It will enable the spider to index your web page faster and can help your ranking.

Avoid using web page creators such as FrontPage, Dreamweaver or a WYSIWYG editor. Software such as that will often times add scripting code that isn't required, making the page larger than it had to be and making it harder to crawl. It will also add code that can not be read by the search engines, causing the spider not to index the page or not index the entire web page. It's much better to use standard HTML. Adding code that they can not read or have a hard time to read can result in significant problems with your position.

Try not to use Flash when possible. Flash can't be read by the search engines to date and will cause download time to slow a little. Should you decide to use Flash anyway, make sure you add text to the webpage, so the search engines have something to read and learn what your webpage is all about. Additionally, it will allow your visitors to have something to read while the Flash file loads. Also don't use Flash as a means of navigation, as I said before spiders can't read Flash.

It is important to add a site map to your web site. Not only will this make it easier for internet surfers to get around your website, but it will also allow spiders to find your website's content easier and index your web page sooner. The site map should contain text links and not image links.

I highly recommend that you take a look at your web page with a Lynx browser because this is similar to how search engines will view your web page. There are other tools online that will enable you to view your webpage.