Bandit based Monte-Carlo planning

Levente Kocsis; Csaba Szepesvári

doi:10.1007/11871842_29

Bandit based Monte-Carlo planning

Levente Kocsis
, Csaba Szepesvári

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

For large state-space Markovian Decision Problems Monte-Carlo planning is one of the few viable approaches to find near-optimal solutions. In this paper we introduce a new algorithm, UCT, that applies bandit ideas to guide Monte-Carlo planning. In finite-horizon or discounted MDPs the algorithm is shown to be consistent and finite sample bounds are derived on the estimation error due to sampling. Experimental results show that in several domains, UCT is significantly more efficient than its alternatives.

Original language	English
Title of host publication	Machine Learning
Subtitle of host publication	ECML 2006 - 17th European Conference on Machine Learning, Proceedings
Pages	282-293
Number of pages	12
DOIs	https://doi.org/10.1007/11871842_29
State	Published - 2006
Externally published	Yes
Event	17th European Conference on Machine Learning, ECML 2006 - Berlin, Germany Duration: 18 Sep 2006 → 22 Sep 2006

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	4212 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th European Conference on Machine Learning, ECML 2006
Country/Territory	Germany
City	Berlin
Period	18/09/06 → 22/09/06

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/11871842_29

Cite this

@inproceedings{df9c8cc011214e34956623f9ac61cbe2,

title = "Bandit based Monte-Carlo planning",

abstract = "For large state-space Markovian Decision Problems Monte-Carlo planning is one of the few viable approaches to find near-optimal solutions. In this paper we introduce a new algorithm, UCT, that applies bandit ideas to guide Monte-Carlo planning. In finite-horizon or discounted MDPs the algorithm is shown to be consistent and finite sample bounds are derived on the estimation error due to sampling. Experimental results show that in several domains, UCT is significantly more efficient than its alternatives.",

author = "Levente Kocsis and Csaba Szepesv{\'a}ri",

year = "2006",

doi = "10.1007/11871842\_29",

language = "אנגלית",

isbn = "354045375X",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "282--293",

booktitle = "Machine Learning",

note = "17th European Conference on Machine Learning, ECML 2006 ; Conference date: 18-09-2006 Through 22-09-2006",

}

Kocsis, L & Szepesvári, C 2006, Bandit based Monte-Carlo planning. in Machine Learning: ECML 2006 - 17th European Conference on Machine Learning, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4212 LNAI, pp. 282-293, 17th European Conference on Machine Learning, ECML 2006, Berlin, Germany, 18/09/06. https://doi.org/10.1007/11871842_29

Bandit based Monte-Carlo planning. / Kocsis, Levente; Szepesvári, Csaba.
Machine Learning: ECML 2006 - 17th European Conference on Machine Learning, Proceedings. 2006. p. 282-293 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4212 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Bandit based Monte-Carlo planning

AU - Kocsis, Levente

AU - Szepesvári, Csaba

PY - 2006

Y1 - 2006

N2 - For large state-space Markovian Decision Problems Monte-Carlo planning is one of the few viable approaches to find near-optimal solutions. In this paper we introduce a new algorithm, UCT, that applies bandit ideas to guide Monte-Carlo planning. In finite-horizon or discounted MDPs the algorithm is shown to be consistent and finite sample bounds are derived on the estimation error due to sampling. Experimental results show that in several domains, UCT is significantly more efficient than its alternatives.

AB - For large state-space Markovian Decision Problems Monte-Carlo planning is one of the few viable approaches to find near-optimal solutions. In this paper we introduce a new algorithm, UCT, that applies bandit ideas to guide Monte-Carlo planning. In finite-horizon or discounted MDPs the algorithm is shown to be consistent and finite sample bounds are derived on the estimation error due to sampling. Experimental results show that in several domains, UCT is significantly more efficient than its alternatives.

UR - https://www.scopus.com/pages/publications/33750293964

U2 - 10.1007/11871842_29

DO - 10.1007/11871842_29

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:33750293964

SN - 354045375X

SN - 9783540453758

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 282

EP - 293

BT - Machine Learning

T2 - 17th European Conference on Machine Learning, ECML 2006

Y2 - 18 September 2006 through 22 September 2006

ER -

Bandit based Monte-Carlo planning

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this