While in-vivo efforts to experimentally test the underlying mechanisms of predictive coding still face enormous challenges, in-silico approaches present an encouraging avenue to study the phenomenon. Recurrent neural networks (RNNs) offer a powerful framework to investigate hypotheses about hierarchical predictive coding, owing to their ability to effectively exploit temporal dependencies typical of natural stimuli, learn distributed representations, and model complex, multi-regional neuronal dynamics. This thesis subsequently employs RNNs as models of cortical circuits, illustrating how they can be leveraged to study the inherent architectural biases of the canonical cortical microcircuit, and their functional implications. Furthermore, this dissertation also develops tools that address limitations concerning the anatomical fidelity of conventional RNN architecture and training, hence improving their overall biological plausibility and utility for modeling neural circuits to gain neuroscientific insights. The contributions of this thesis therefore provide a robust, computational framework for modelling and interpreting complex, multi-regional neuronal data, ultimately paving the way for deeper insights into how the structural properties of cortical circuits support sophisticated information processing and intelligent behaviour.