Investigating Building Blocks of Cognition in Realistic Reinforcement Learning Environments

Abstract

The convergence of Deep Reinforcement Learning (DRL) and cognitive neuroscience has opened avenues for probing neural representations within the brain. This thesis delves into the synergy between decision-making and representation by harnessing DRL’s capabilities. Through a focus on dot motion perceptual decision-making task within a high-dimensional framework akin to psychological experiments, we acquire novel insights into how intricate neural networks tackle complex tasks. The resultant end-to-end model not only elucidates network strategies, but also offers a backdrop for understanding analogous brain processes. Investigation reveals intriguing resemblances between the network’s behavior and neural mechanisms observed in the middle temporal visual area (MT) of the brain, known for encoding direction and motion strength. Remarkably, direction selectivity emerges solely through reward-driven training, while graded firing patterns encode motion strength. A testable hypothesis arises, suggesting coherenceselective neurons within the MT population. While conventional Reinforcement Learning (RL) presupposes uniform data distributions, real-world scenarios like autonomous driving and natural environments display Zipfian distributions, characterized by frequent occurrences amidst rare events. Inspired by complementary learning systems theory, this work introduces an architecture tailored for learning from Zipfian distributions. Unsupervised discovery of long tail states and their retention in episodic memory, coupled with recurrent activations, forms the core of the proposed architecture. Retrieval from episodic memory via similarity-based search, reinforced with weighted importance, amplifies performance across diverse Zipfian tasks. Our proposed architecture achieves higher accuracy than the IMPALA algorithm across all three tasks and evaluation metrics (Zipfian, Uniform, and Rare Accuracy). This thesis advances the nexus of decision-making and neural representation through DRL, while proposing an innovative architecture capable of navigating the intricacies of Zipfian data distributions. It not only augments our grasp of cognitive building blocks but also bears implications for resolving challenges across real-world domains. This research in one small step that bridges the gap between computational models and cognitive processes, opening new vistas for understanding and application.