Collection Selection and Results Merging With Topically Organized U.S. Patents and TREC Data
MASSACHUSETTS UNIV AMHERST DEPT OF COMPUTER SCIENCE
Pagination or Media Count:
We investigate three issues in distributed information retrieval, considering both TREC data and U.S. Patents 1 topical organization of large text collections, 2 collection ranking and selection with topically organized collections 3 results merging, particularly document score normalization, with topically organized collections. We find that it is better to organize collections topically, and that topical collections can be well ranked using either INQUERYs CORI algorithm, or the Kullback-Leibler divergence KL, but KL is far worse than CORI for non-topically organized collections. For results merging, collections organized by topic require global idfs for the best performance. Contrary to results found elsewhere, normalized scores are not as good as global idfs for merging when the collections are topically organized.
- Information Science