The opposite of smoothing: A language model approach to ranking query-specific document clusters

Exploiting information induced from (query-specific) clustering of top-retrieved docu- ments has long been proposed as a means for improving precision at the very top ranks of the returned results. We present a novel language model approach to ranking query-specific clusters by the presumed percentage of relevant documents that they contain. While most previous cluster ranking approaches focus on the cluster as a whole, our model utilizes also information induced from documents associated with the cluster. Our model substan- tially outperforms previous approaches for identifying clusters containing a high relevant- document percentage. Furthermore, using the model to produce document ranking yields precision-at-top-ranks performance that is consistently better than that of the initial rank- ing upon which clustering is performed. The performance also favorably compares with that of a state-of-the-art pseudo-feedback-based retrieval method.