Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning

Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning.
Peter R. Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J. Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Varun Kompella, Piyush Khandelwal, HaoChih Lin, Patrick MacAlpine, Declan Oller, Craig Sherstan, Takuma Seno, Michael D. Thomure, Houmehr Aghabozorgi, Leon Barrett, Rory Douglas, Dion Whitehead, Peter Duerr, Peter Stone, Michael Spranger, and and Hiroaki Kitano.
Nature, 62:223–28, Feb. 2022.
Available from Nature website.
project webpage

Download

[PDF]2.7MB  

Abstract

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits1. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world's best e-sports drivers. We combine state-of-the-art, model-free, deep reinforcement learning algorithms with mixed-scenario training to learn an integrated control policy that combines exceptional speed with impressive tactics. In addition, we construct a reward function that enables the agent to be competitive while adhering to racing's important, but under-specified, sportsmanship rules. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world's best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges of using these techniques to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

BibTeX Entry

@article{nature22,
  author="Peter R.\ Wurman and Samuel Barrett and Kenta Kawamoto and James MacGlashan and Kaushik Subramanian and Thomas J.\ Walsh and Roberto Capobianco and Alisa Devlic and Franziska Eckert and Florian Fuchs and Leilani Gilpin and Varun Kompella and Piyush Khandelwal and HaoChih Lin and Patrick MacAlpine and Declan Oller and Craig Sherstan and Takuma Seno and Michael D.\ Thomure and Houmehr Aghabozorgi and Leon Barrett and Rory Douglas and Dion Whitehead and Peter Duerr and Peter Stone and Michael Spranger and and Hiroaki Kitano",
  title="Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning",
  journal="Nature",
  year={2022},month="Feb.",day="10",
  pages="223--28",
  volume="62",
  issue="7896",
  doi={10.1038/s41586-021-04357-7},
  abstract="Many potential applications of artificial intelligence
            involve making real-time decisions in physical systems
            while interacting with humans. Automobile racing
            represents an extreme example of these conditions; drivers
            must execute complex tactical manoeuvres to pass or block
            opponents while operating their vehicles at their traction
            limits1. Racing simulations, such as the PlayStation game
            Gran Turismo, faithfully reproduce the non-linear control
            challenges of real race cars while also encapsulating the
            complex multi-agent interactions. Here we describe how we
            trained agents for Gran Turismo that can compete with the
            world's best e-sports drivers. We combine
            state-of-the-art, model-free, deep reinforcement learning
            algorithms with mixed-scenario training to learn an
            integrated control policy that combines exceptional speed
            with impressive tactics. In addition, we construct a
            reward function that enables the agent to be competitive
            while adhering to racing's important, but under-specified,
            sportsmanship rules. We demonstrate the capabilities of
            our agent, Gran Turismo Sophy, by winning a head-to-head
            competition against four of the world's best Gran Turismo
            drivers. By describing how we trained championship-level
            racers, we demonstrate the possibilities and challenges of
            using these techniques to control complex dynamical
            systems in domains where agents must respect imprecisely
            defined human norms.",
  wwwnote={Available from <a href="https://www.nature.com/articles/s41586-021-04357-7?">Nature website</a>.<br>
           <a href="https://www.gran-turismo.com/us/gran-turismo-sophy/">project webpage</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Mar 25, 2024 00:05:09