Super Turing Evolving Lifelong Learning Architecture (STELLAR)

The overarching goal of HRL's Super Turing Lifelong Learning ARchitecture (STELLAR) project was to build a general-purpose scalable autonomous system that continually improves its performance during deployment, and rapidly and safely adapts to new tasks and circumstances, without catastrophic forgetting or the complete undoing of previously learned tasks. Human brains are the best available examples of lifelong learners. Leveraging HRL Teams long-standing expertise in neuroscience, cognitive architectures, machine learning, neuroevolution, and robotics, STELLAR integrates several brain-inspired mechanisms that operate across multiple spatial and temporal scales. These include memory consolidation or stabilization of memories during sleep, neuromodulation, which regulates neural activities and synaptic plasticity, adult neurogenesis or the birth of neurons, instincts, and reflexes. During Phase 1, we developed nine innovative components that solve different challenges and requirements of lifelong learning. During Phase 2, we integrated these components into a complete lifelong learning system and performed rigorous testing and evaluation. In particular, we demonstrated the efficacy of the fully integrated STELLAR system for the autonomous driving domain in the CARLA simulator with online adaptation to different weather conditions, different vehicle models (wheel asymmetries, vehicle dynamics), and different driving tasks (driving in the correct lane vs. opposite lane in regular traffic). The STELLAR system achieved significant performance improvements relative to conventional machine learning Single Task Experts, exceeding key program targets: about 25 performance improvement, about 7.5x increase in learning efficiency, and about 3.5x improvement in Forward Transfer (leveraging prior learning to facilitate learning a new task).