Search Engines Robots

Search Engines — How They See Your Website
by Maggi Hall, E-Commerce Adviser staff (rev022508)

A well designed web page engages a visitor and helps convert a visitor into a customer. A well designed web page must also be search engine "friendly" because most e-commerce businesses rely on search engines to generate traffic.

Robots & Spiders

. Search engines use "robots," also called "spiders," to view web pages. The robots go from page to page on a website collecting information. But the information collected by a robot may not be the same as what a visitor sees. For example, robots:

Do not see the content of pictures
Do not perform searches, such as a visitor might do to see products on Amazon.com or Ebay

This means that a web page, which is great for visitors, could be a disaster for a search engine. To avoid a disaster you want to use a website technology that is search engine compatible without compromising the visitor's experience.

Web Page Examples

. To illustrate the difference between visitors and robots, here are links to two actual web pages — Sample A and Sample B. These pages look very similar to visitors, however, are quite different technically and are viewed very differently by robots.

The "Search Engine" links below show the information seen by a search engine on each sample page. Without even looking closely at the content of each sample, there is clearly a difference in quantity of text between the two pages. If you had selected the Sample B technology, search engines would have very little information about your website content.

Sample A -- Search Engine View Sample B -- Search Engine View

Technical Design

. Selecting a technical design for your website should be driven by the website objectives. It also is determined by your budget. Sometimes that means working with a professional designer or developer. If you are using a professional to build your website, be sure to include your search engines objectives in the design discussions. Do not assume the developer will automatically build a website that is "search engine friendly". In fact, frequently professionals like to use the latest technology, which search engines do not understand.

Many small business owners decide to build their own website even though they have limited technical understanding. Generally this is fine. Less technical people tend to create simpler web pages, such as plain HTML, which search engines like. But sometimes business owners use templates or "canned" websites, which may not be "search engine friendly."

If you are thinking about building your own website and you are not very technical, you may want to research your planned design before starting. To evaluate a prospective design, you can try posting something on a forum or finding a technical mentor.

Related Articles