Towards Robust Reasoning in Neural Networks Through Mechanistic and Functional Perspectives

Artificial Neural Networks are frequently referenced as black-box decision systems - while they are performant on many tasks across data modalities, current understanding of the mechanisms underpinning their success is limited. As a result, when considering the remarkable performance of neural networks in conjunction with critical societal questions such as bias, fairness and trustworthiness, neural network opacity stands as a bottleneck for safe adoption. The nascent field of Mechanistic Interpretability (MI) posits the philosophy that the fundamental units of computation in neural networks are comprised of features - these features are learned via loss minimisation and are combined to create circuitry behavior that processes inputs. As a result, such circuits can be extracted, modified and reversed-engineered to understand how artificial neural networks (ANNs) handle inputs. A renewed hope stems from this perspective and promises to slowly decode ANN's latent encoding. While the literature has shown that circuitry behavior exists within toy models and state-of-the-art models alike, notions of polysemantic neurons and phenomena such as superposition, enabled by non-linear activation functions, stand in the way of full model interpretability. Subsequently, fundamental questions arise from this line of inquiry, this research stands to answer questions regarding how circuit formation occurs and relate it to existing theories in deep learning. Further we explore the Functional Perspective of DNNs which hopes to serve as tool to further develop MI research and address some of the identified weaknesses of current MI literature as well as moving towards answering the open research questions in the field. Finally potential research avenues are introduced alongside proposed experimental setups to test hypotheses that aim to begin answering the questions outlined; with the relation to safety considerations inline with the UKRI Safe and Trusted AI CDT.