|
The Search Engine Tutorial for Web Designers explains, how to design your pages, keeping the search engines in mind, and why it is necessary to do so. Knowledge is a powerful weapon and having knowledge of what a search engine is going to do with your page, while you are designing it, allows you to build a better resulting web page.
The Search Engine Tutorial for Web Designers was born from the need to understand why one of our best web sites fared so poorly in the search engines. In many cases we have applied the changes we suggest here and have seen improvements in our results. Our quest began when we started to examine the reasoning behind why our Beginners' Central was so badly listed in the search engines. Beginners' Central is an award winning tutorial for new internet users and had been developed over the course of many months. With ongoing development, it was possible to attempt to redress the search engine issue many times, and in the process, learning what worked and what didn't. This document will share with you what Northern Webs has learned about designing "Search Engine Friendly" web pages.
The biggest problem that most people have with the search engines, is a failure to understand what the Search Engines are going to do with each page they have designed. Because of this failure to understand the search engines, there are many fantastically beautiful websites out there which will never recieve the exposure they deserve. Unlike a directory, the search engines only allow you to enter a url for submission to their system. How a search engine indexes and summarizes your web page is entirely out of your control. Or is it?
The truth be known, its not out of your control, it's entirely up to you, but like the many different search engines, there is more than one method which you will need to employ to accomplish this.
This document is broken down into major sections. Each major search engine is detailed, containing annotated examples of what the search engine does as well as links to various sections within search engine itself. The Design Notes section contains information specific to html requirements for proper design, links to useful resources and of course our Meta Medic(tm).
Definition of terms
For every search engine we talk about here, we will have a brief table highlighting some of the more important facts about that engine. It is for this reason we have listed a brief description of the terms below so you may familiarize yourself with them before digging into the material.
Size - The approximate amount of urls contained within the search engine index.
Spider Class - There are two spider classes, Deep and Shallow. A deep spider will take a url and spider all of the pages within the site, no matter how many levels of directories it needs to traverse. A shallow spider can do one of two things, it can either spider the url given and stop, or only spider those urls it finds within a single level of directories.
Meta Tag support - Whether or not a search engine will deal appropriately with the meta tags in the html. Values for this field can be Yes/No/Partial.
Frame support - Can the search engine spider a site which relies soley on the frameset urls (Yes/No).
Image Map support - Can the search engine read the urls embedded within a client side imagemap(Yes/No)?
Alt Text support - Does the search engine also index text contained within the Alt modifier to an image statement(Yes/No)?
HTML Comments - Does the search engine read and index html comments(Yes/No)?
Url Searching - Does the search engine have the ability to search given a url(Yes/No)?
Embedded Directory - Does the Search Engine contain an embedded directory in their site(Yes/No)?
Submission URL - This will be the url you need to submit your url to this search engine.
SEARCH ENGINE Vs DIRECTORY SYSTEM
This is an issue which confuses a lot of people. so perhaps it would be best to define the primary differences here and now, thus clearing up any confusion you might have.
Search Engine - A search engine is a database system designed to index internet addresses (urls, usenet, ftp, image locations etc). The typical search engine contains a special program often called a spider (also sometimes called a "bot" or "crawler"), the spider accepts a url, it then goes to that website and retrieves a copy of the file found there. Sometime later, it the search engine will process that copy of the file, distilling it down to the bare essential data it needs for the data base. While most search engines request both a url and an email address, the search engine makes the determination as to what data ends up in the database. In short, given a url, an automated process occurs which results in your site being included into the index.
Directories - A directory is basically a manual entry database system. You, as the end user submitting your url, will supply the directory with all of the needed information during the submission process. At a minimum, this information includes, url, title and a short summary of your website. Rarely will the directory have any program capable of visiting your website, although a few directories do have a simple spider capable of verifying that the url you provided was a valid url.
While both classes of systems have elements in common, such as the ability to search the data base, boolean expressions, and advanced features. The primary distinction lies in how the two systems obtain their data. One does it automatically (search engines) and the other does it manually (directories).
BANNING
Because of a tendency for people to utilize anything free, and considering the statements about blocking all pages within a domain made by most Search Engines, we have checked several of the more common free webspace servers to see if they are adequately represented within the index of each engine.
A free webspace server is a system which will host your site at no cost to you. They can allow this because their primary source of income is generated by reserving a small piece of screen real estate for themselves, on which they can display banners.
Freespace servers often take urls in the following formats:
http://mysite.freespaceserver.com
http://www.freespaceserver.com/directory/111111
http://www.myname.freespaceserve.com
If you're building a commercial website, that is, a site that is selling something, and your budget allows for it, we urge you to consider obtaining a domain name and hosting at a regular hosting company instead of a freespace server. However if you're limited by your budget, a freespace server can serve as a suitable stepping stone until you can afford a regular host.
If you're going to host at a free server, we strongly suggest that you pick a server that does not encapsulate your site within a frame. Frames can severely limit your chances of being located within the search engines if they are not designed correctly, and in many cases, free space servers using frames do not give you the ability to edit the frameset file.
Yes=Most likely banned. No=Not banned from the search engine. N/A - The Search Engine does not provide a url count for search results
Table Contents last updated Tuesday, March 14, 2000
We define "Adequate Representation" as to mean containing 10,000 pages or more from these domains.
The results were obtained by searching on the base url, ie. "geocities.com" or "hypermart.net" and observing the number of urls the search engine returned. Since these free webpage systems claim to have hundreds of thousands of active urls, having less than 10,000 urls indexed would be an inadequate representation within that search engine.
The issue here is clear and straightforward. If you are planning to host your site on a freespace server, we suggest picking one that is well represented in the search engines:
SET Home
Alta Vista
InfoSeek
HotBot
Lycos
Northern Light
Excite
Web Crawler Design Notes
Summary
Directories
Useful Links
Kudos received
Meta Medic
Webmaster Tools
|
