Here is a summary of Inktomi's announcement. For a cost of $20 per year for the first unique URL, with the price dropping for subsequent URL's, Inktomi will guarantee that their spider will come and spider that URL every 48 hours. Note, this is unique URL's, which means you would have to pay for every page on your site that you want included in Inktomi's index. For our site, which has about 200 unique pages, the cost would be somewhere around $1200 per year. That's the price we would pay to make sure we're included in Inktomi's index. There were precious few additional details in Inktomi's announcement, which was released through a strategic partner that will be handling the paid submission program, Positiontech. According to Inktomi, Positiontech is just the first of many partners where the paid submission will be available, including Network Solutions.

Inktomi's Background

For the uninitiated, Inktomi is a pioneer of search engine technology that has fallen on hard times recently. Their search technology has been private branded and has powered some of the most popular search portals on the web, including Yahoo, Hotbot, Goto, Excite, MSN and others. While they still have several strategic partnerships, making them a significant player with the search engines, their importance was diminished greatly when Yahoo dumped Inktomi as their index based search provider earlier this year (see article) and went with Google instead. The resulting hit on Inktomi's share prices and the shake up of confidence in their ability to produce revenue left Inktomi looking for other places to turn a buck. Apparently, their decision was to turn to the website owners and ask them to shell out for the privilege of being included in Inktomi's index.

Going Down a Dangerous Path

Inktomi is heading down a road where other search engines have been burned before. The whole idea of search engines requiring site owners to pay to achieve top rankings is one that generates fierce controversy on the Net. Last year, AltaVista tried introducing a paid results program that would ensure that sponsor's links would prominently appear on top of the free listings. After a flurry of complaints from both advertisers and users, AltaVista quietly dropped the program.

MSN tried a similar tact this year and also had to withdraw the program after a few months. Google is currently trying something similar, although more discretely, and the success of this attempt remains to be seen.

What AltaVista and MSN quickly learned is that the users of their services largely ignored anything that smacked of being a paid return. Click throughs from the paid rankings were abysmal. Advertisers simply didn't get their money's worth.

GoTo-A-GoGo

The one success story in the pay per click sector is Goto. They make no bones about the fact that advertisers bid to get the top ranking. In fact, you can even see how much they've paid to stake their claim to that ranking. (More than one web commando has looked up their competition on Goto and gleefully clicked away, running up the unfortunate site owner's advertising tab. Petty? Absolutely. Perversely satisfying? You betcha!). Yet, unlike AltaVista and MSN, Goto's users seem to accept the paid listings without a quibble. Goto's traffic numbers continue to increase and many of the clients we have that use Goto report good click throughs and conversion rates. More importantly to Goto's shareholders, it's probably the only search engine actually making money.

Inktomi's Pay to Stay Strategy

Perhaps fearing to tread in the footsteps of AltaVista and MSN, Inktomi is trying a different revenue generation tact. Rather that auctioning off top spots and charging per click, Inktomi is offering a spidering guarantee. Inktomi is charging to guarantee that their spider will index your page on a regular basis. This is somewhat similar to Yahoo's or Looksmart's business express submission, where you pay just to get an editor to come and review your site. There's no guarantee of placement or inclusion. You're just ensuring you'll be looked at and considered.

At this point, Inktomi is still offering their free Submit URL service. However, it's very likely that it will suffer from the same quirks that has plagued their index for the past several months. We've found that while Inktomi will index and include most pages submitted, all but the main page of the site will be dropped from the index after a few weeks. One must assume that the paid submission program is Inktomi's answer on how to get pages to stick in it's index.

Is It Worth It?

Putting aside the more fundamental question of whether it's right for Inktomi to do this for now, the question most webmasters are asking is if it's worth it? Can Inktomi generate enough traffic to make the cost worthwhile? Probably not.

Based on our clients search engine traffic, visitors that come directly from an Inktomi based engine account for about 12% of all unique visits. Stack this up against Google, which generates about 23% of all traffic (including their Yahoo partnership) and doesn't charge a dime to be included in their index. Or AltaVista at 15%, also at no cost. Even Yahoo, who initially got a lot of flack for their business express submission, looks like a bargain compared to Inktomi. Yahoo, for the directory alone, generates 27% of all search engine traffic. And, if you get accepted the first time, you only pay the $199 submission once.

Unless you have a large online marketing budget, I would say your Inktomi money could be better spent elsewhere.

But Is It Right?

Nothing better to wrap up a column than a little moral condemnation. The whole purpose of a search engine is to present you, the user, with quality sites that are most relevant to your search, not the sites that have the biggest advertising budget. While Goto has been a profitable venture, this has always been my complaint against it. It's the reason my preferred search choices have been AltaVista and, more recently, Google. Apparently, my feelings have been shared because while Goto's traffic has steadily increased over the past year, Google's has sky rocketed.

By drawing this line in the sand, Inktomi is saying that only site owners with big budgets have a chance of being included in their index. They've shut the door on the huge majority of site owners. The only real asset Inktomi has to offer is the quality of their index and by taking this step, they're devaluing that asset substantially. In my opinion, this will hurt Inktomi in the long run, perhaps fatally. If I'm wrong, Inktomi will have fundamentally changed the way search engines generate revenue and others will follow. The Net will become a place where only the richest site owners survive. And as much as I'd hate to see Inktomi fade away, I'd much prefer this to the alternative.

That said, let me end by recognizing the financial realities of the Search Engine biz. While search engines like Google and AltaVista may be sticking to the idealistic goals of providing unbiased search results and selling advertising separately, they do have to make money. Selling advertising tied to search results just hasn't been successful enough allow these companies to turn a profit. Inktomi was struggling to survive. They needed another way to generate revenue. Did they make the right decision? The jury is out, and that jury consists of millions and millions of site owners and users. We'll see, Inktomi, we'll see.

More on Inktomi:

Inktomi instability
(July 25, 2000)

Inktomi Developments
(August 22, 2000)

Inktomi's Paid Inclusion Program
(Nov. 02, 2000)

More on paid submissions:

What are my Paid Submission options??BR>(Oct. 27, 2000)

All About Looksmart
(Oct. 31, 2000)