A comparative analysis of human and automatic query variants

Binsheng Liu; Nick Craswell; Xiaolu Lu; Oren Kurland; J. Shane Culpepper

doi:10.1145/3341981.3344223

A comparative analysis of human and automatic query variants

Binsheng Liu, Nick Craswell, Xiaolu Lu, Oren Kurland, J. Shane Culpepper

Data and Decision Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We present an in-depth comparative analysis of the effectiveness distributions of sets of human-created and automatically-created query variations used to represent the same information need. The automatic variations are generated using Bing's click graph. Experiments performed with TREC datasets show that using automatic variations for retrieval can result in similar effectiveness to that of using human variations, although the two types of variations can be appreciably di?erent in several important respects - e.g., their similarities and corresponding retrieved lists.

Original language	English
Title of host publication	ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval
Pages	47-50
Number of pages	4
ISBN (Electronic)	9781450368810
DOIs	https://doi.org/10.1145/3341981.3344223
State	Published - 23 Sep 2019
Event	9th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2019 - Santa Clara, United States Duration: 2 Oct 2019 → 5 Oct 2019

Publication series

Name	ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval

Conference

Conference	9th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2019
Country/Territory	United States
City	Santa Clara
Period	2/10/19 → 5/10/19

ASJC Scopus subject areas

Computer Science (miscellaneous)
Information Systems

Access to Document

10.1145/3341981.3344223

Cite this

Liu, B., Craswell, N., Lu, X., Kurland, O., & Culpepper, J. S. (2019). A comparative analysis of human and automatic query variants. In ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval (pp. 47-50). (ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval). https://doi.org/10.1145/3341981.3344223

@inproceedings{c69824896232483abbf070f482223e97,

title = "A comparative analysis of human and automatic query variants",

abstract = "We present an in-depth comparative analysis of the effectiveness distributions of sets of human-created and automatically-created query variations used to represent the same information need. The automatic variations are generated using Bing's click graph. Experiments performed with TREC datasets show that using automatic variations for retrieval can result in similar effectiveness to that of using human variations, although the two types of variations can be appreciably di?erent in several important respects - e.g., their similarities and corresponding retrieved lists.",

author = "Binsheng Liu and Nick Craswell and Xiaolu Lu and Oren Kurland and Culpepper, {J. Shane}",

note = "Publisher Copyright: {\textcopyright} 2019 Copyright held by the owner/author(s).; 9th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2019 ; Conference date: 02-10-2019 Through 05-10-2019",

year = "2019",

month = sep,

day = "23",

doi = "10.1145/3341981.3344223",

language = "אנגלית",

series = "ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval",

pages = "47--50",

booktitle = "ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval",

}

Liu, B, Craswell, N, Lu, X, Kurland, O & Culpepper, JS 2019, A comparative analysis of human and automatic query variants. in ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval. ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, pp. 47-50, 9th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2019, Santa Clara, United States, 2/10/19. https://doi.org/10.1145/3341981.3344223

A comparative analysis of human and automatic query variants. / Liu, Binsheng; Craswell, Nick; Lu, Xiaolu et al.
ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval. 2019. p. 47-50 (ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A comparative analysis of human and automatic query variants

AU - Liu, Binsheng

AU - Craswell, Nick

AU - Lu, Xiaolu

AU - Kurland, Oren

AU - Culpepper, J. Shane

PY - 2019/9/23

Y1 - 2019/9/23

N2 - We present an in-depth comparative analysis of the effectiveness distributions of sets of human-created and automatically-created query variations used to represent the same information need. The automatic variations are generated using Bing's click graph. Experiments performed with TREC datasets show that using automatic variations for retrieval can result in similar effectiveness to that of using human variations, although the two types of variations can be appreciably di?erent in several important respects - e.g., their similarities and corresponding retrieved lists.

AB - We present an in-depth comparative analysis of the effectiveness distributions of sets of human-created and automatically-created query variations used to represent the same information need. The automatic variations are generated using Bing's click graph. Experiments performed with TREC datasets show that using automatic variations for retrieval can result in similar effectiveness to that of using human variations, although the two types of variations can be appreciably di?erent in several important respects - e.g., their similarities and corresponding retrieved lists.

UR - http://www.scopus.com/inward/record.url?scp=85074229463&partnerID=8YFLogxK

U2 - 10.1145/3341981.3344223

DO - 10.1145/3341981.3344223

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85074229463

T3 - ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval

SP - 47

EP - 50

BT - ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval

T2 - 9th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2019

Y2 - 2 October 2019 through 5 October 2019

ER -

Liu B, Craswell N, Lu X, Kurland O, Culpepper JS. A comparative analysis of human and automatic query variants. In ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval. 2019. p. 47-50. (ICTIR 2019 - Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval). doi: 10.1145/3341981.3344223

A comparative analysis of human and automatic query variants

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this