As early as 1996, a proposal was written to create a META tag to be used within the HTML of a page. Adding "metadata" tags to the HTML structure of the day for the purpose of better describing web pages. By then, hundreds of thousands of web pages had already been written, and the concern was that while pages could be indexed, there needed to be something that would allow to better automate their indexing.

That was known as the Proposal. Back in early 1999, papers were finally submitted to the World Wide Web Consortium - http://www.w3.org/ - a Resource Description Framework, but the tag had been in use for years.

Just as a library has a card index to catalog the millions of books, metadata was created to assist in creating the same thing for web pages. If you don't know what a Meta Tag looks like, try looking at the description at the W3 site. It is the purest form of information on HTML data.

For years webmasters were diligent including the necessary data into their pages. In fact, when you "View Source" on most web pages today you can see the meta data included within the HEAD tag.

If some of this is way over your head, hang in there; we're almost to the point.

Search engines have been around since 1993. The dream was to enable their "bots" to spider the web more effectively by indexing using something (metadata) within web pages. As early as 1994, search engine bots were already indexing entire pages. Because of the popularity of whole page indexing, sites such as "Webcrawler", later purchased by AOL, could only be used at night. During the day the number of searches simply crashed the servers.

Infoseek and AltaVista were the first to use the keywords meta tag in 1996. HotBot and Lycos soon followed. But what soon became apparent in 1997 was that some web site owners would insert misleading words about their pages, and some would use excessive repetition of words in hopes of tricking the search engines into thinking their pages were more relevant (read: Spamming).

Lycos stopped supporting the tag in 1998, and newer search engines such as Google and FAST never added support at all. When Infoseek ended in 2000, only AltaVista and Inktomi remained. AltaVista then joined the crowd and stopped supporting the keyword meta tag.

"In the past we have indexed the meta keywords tag but have found that the high incidence of keyword repetition and spam made it an unreliable indication of site content and quality. We do continue to look at this issue, and may re-include them if the perceived quality improves over time," said Jon Glick, AltaVista's director of internet search.

Not sure why Inktomi hangs in there. Their explanation is:

"The meta keywords value is just one of many factors in our ranking equation, and we've never given too much weight to it. That said, we will continue to use it as long as our relevance modeling shows that it adds value," Ken Norton, director of product marketing for Inktomi's web search division.

So if by 1994 we already had contextual based search engines, why in 1999 did we need metadata?

Well, that's just the point. We didn't really need them for search engine inclusion. As a matter of fact, the W3 explains the main reason would be to notify search engines that your site contains pages in another language, and where those pages are located. But as far as using metadata to get included in search engines, W3 simply touches on the Keyword and Description meta tags. Their focus is on better structure of the web page, content, and improved use of the robots.txt file.

Okay, so we learned that we don't need Meta Tags for the search engines to find us. And that since the early '90's the "bots" have been spidering the web, including pages into various collections. But we do know that a higher ranking in the search engines will improve our AdSense performance. In other words, more people visiting your site, the greater the probability they will click your AdSense ads and make you some money.

The fallacy is that souping your pages up with (stuffing) keywords will increase your search engine rank. Guess what? Nobody uses Inktomi.

Founder of one of the first web hosting companies on the Internet, World Wide Mart, Dave Jackson has been teaching others how to monetize their web sites for over ten years. Currently he teaches others to create a solid residual income by creating quality web sites of great benefit to visitors and monetizing them using contextual ads. His blog is Making Money with Contextual Ads.