Matthew E. Taylor's Publications

Sorted by DateClassified by Publication TypeSorted by First Author Last NameClassified by Research Category

Adaptive Tile Coding for Value Function Approximation

Shimon Whiteson, Matthew E. Taylor, and Peter Stone. Adaptive Tile Coding for Value Function Approximation. Technical Report AI-TR-07-339, University of Texas at Austin, 2007.

Download

[PDF]329.4kB  [postscript]942.5kB  

Abstract

Reinforcement learning problems are commonly tackled by estimating the optimal value function. In many real-world problems, learning this value function requires a function approximator, which maps states to values via a parameterized function. In practice, the success of function approximators depends on the ability of the human designer to select an appropriate representation for the value function. This paper presents adaptive tile coding, a novel method that automates this design process for tile coding, a popular function approximator, by beginning with a simple representation with few tiles and refining it during learning by splitting existing tiles into smaller ones. In addition to automatically discovering effective representations, this approach provides a natural way to reduce the function approximator's level of generalization over time. Empirical results in multiple domains compare two different criteria for deciding which tiles to split and verify that adaptive tile coding can automatically discover effective representations and that its speed of learning is competitive with the best fixed representations.

BibTeX Entry

@TechReport{whitesontr07,
  author       = "Shimon Whiteson and Matthew E. Taylor and Peter Stone",
  title	       = "Adaptive Tile Coding for Value Function Approximation",
  institution  = "University of Texas at Austin",
  number       = "AI-TR-07-339",
  year	       = 2007,
  abstract={Reinforcement learning problems are commonly tackled by
   estimating the optimal value function. In many real-world problems,
   learning this value function requires a function approximator,
   which maps states to values via a parameterized function. In
   practice, the success of function approximators depends on the
   ability of the human designer to select an appropriate
   representation for the value function. This paper presents
   \emph{adaptive tile coding}, a novel method that automates this
   design process for tile coding, a popular function approximator, by
   beginning with a simple representation with few tiles and refining
   it during learning by splitting existing tiles into smaller
   ones. In addition to automatically discovering effective
   representations, this approach provides a natural way to reduce the
   function approximator's level of generalization over
   time. Empirical results in multiple domains compare two different
   criteria for deciding which tiles to split and verify that adaptive
   tile coding can automatically discover effective representations
   and that its speed of learning is competitive with the best fixed
   representations.},
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Apr 19, 2010 14:12:46