Learning to Co-operate in Multi-Agent Systems Experiments with the RoboCup Simulator

In recent years, two major areas of computer science§have started converging. Artificial intelligence§research is moving towards realistic domains§requiring real-time responses, and real-time systems§are moving towards more complex applications§requiring intelligent behaviour. This book addresses§the question of whether agents can learn to become§individually skilled and also learn to co-operate in§the presence of both teammates and adversaries in a§complex, real-time, noisy environment with no§communication.§ To answer this question this work starts by§presenting a multi-threaded agent architecture§capable of dealing with the logical and timing§challenges of such an environment. The decision§making process is broken down into simple modules§that link together an agent s perception to its§actions. The book demonstrates how a sparse§distributed memory model can be used as a§generalisation component for tasks that involve large§state spaces. It further demonstrates how§reinforcement learning can be linked to such a memory§model and produce intelligent action. Experimental§results demonstrate how a learned policy can§outperform fixed, hand-coded ones. In recent years, two major areas of computer science§have started converging. Artificial intelligence§research is moving towards realistic domains§requiring real-time responses, and real-time systems§are moving towards more complex applications§requiring intelligent behaviour. This book addresses§the question of whether agents can learn to become§individually skilled and also learn to co-operate in§the presence of both teammates and adversaries in a§complex, real-time, noisy environment with no§communication.§To answer this question this work starts by§presenting a multi-threaded agent architecture§capable of dealing with the logical and timing§challenges of such an environment. The decision§making process is broken down into simple modules§that link together an agent s perception to its§actions. The book demonstrates how a sparse§distributed memory model can be used as a§generalisation component for tasks that involve large§state spaces. It further demonstrates how§reinforcement learning can be linked to such a memory§model and produce intelligent action. Experimental§results demonstrate how a learned policy can§outperform fixed, hand-coded ones.