Introduction
Equipping robots with the ability to learn a growing set of tasks over their operational lifetime—rather than focusing on mastering individual tasks—presents a significant challenge in robot learning.
Lifelong learning robots often struggle with catastrophic forgetting when learning from changing input distributions, which causes the robot to forget old knowledge when learning new tasks. They are also expected to leverage previous knowledge to accelerate learning of new tasks without the need for complete retraining. This is often referred to as the stability-plasticity dilemma, where stability denotes the retention of old knowledge, and plasticity refers to the acquisition of new knowledge.
Recent advancements in continual lifelong learning have proposed three primary strategies to address this dilemma: regularization, dynamic growth, and experience replay [1]. However, these methods typically demand high storage and computational resources, leading to increased energy consumption for data storage, processing, and transmission. Additionally, robots often face limitations in onboard resources, making it difficult to support lifelong learning outside controlled lab environments and to retain and integrate experiences from various environments and tasks. Although some recent approaches have shown promise in improving efficiency in continual robot learning [2, 3], they often come at the cost of reduced performance compared to single-task models, where each task is learned with a dedicated model.
This project aims to develop a continual on-device robot learning system that improves the trade-off between stability and plasticity while enhancing resource efficiency without compromising performance.
The system will be designed for deployment on resource-constrained, non-networked robotic platforms and aims to contribute to sustainability by reducing carbon emissions through improved operational efficiency, including minimizing the need for frequent retraining and optimizing data handling processes.
References:
References
[1] Wang, L., Zhang, X., Su, H. and Zhu, J., 2024. A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence.
[2] Hafez, M.B., Immisch, T., Weber, T. and Wermter, S., 2023. Map-based experience replay: a memory-efficient solution to catastrophic forgetting in reinforcement learning. Frontiers in Neurorobotics, 17, p.1127642.
[3] Schwarz, J., Czarnecki, W., Luketina, J., Grabska-Barwinska, A., Teh, Y.W., Pascanu, R. and Hadsell, R., 2018, July. Progress & compress: A scalable framework for continual learning. In International conference on machine learning (pp. 4528-4537). PMLR.