ZDS

Search Engines

Mark III is being programmed now.

It's features are a direct result of the experience gained with the first two.

Vtune analysis of Mark II shows that 98 percent of the index server run time is IO.

The actual IO involved is very minor (See Mark II graphic IO budget sidebar), but the server has to open a minimum of three files 40 percent of the time, and five files for the other 60 percent. File opens are very expensive.

Mark III will require only one file open.

Mark I is limited by the size of the executable. For large page sets the time spent loading the entire index into memory, eventually will cost more than the time to load the (much) smaller basic index server, and open and read separate index files for the specific information.

The Mark II index is about 30% of the original data. It was designed for speed and space efficiency was secondary. The bit map file contains about 50 percent empty pages and declaring it as a sparse files cuts this accordingly, but total index space usage as a percentage of the original database is still too high.

The goal for Mark III index space is 10-15 percent of the original data.