Neuroimaging research is starting to move from linear machine learning methods towards neural networks. Since neural networks can fit more flexible functions to data, they are a prime candidate for finding relationships that linear machine learning methods are unable to find. However, the most successful applications of data-driven linear machine learning methods, like independent component analysis, to neuroimaging data used neurophysiological priors about whole-brain spatiotemporal dynamics in order to capture interpretable features that provided new insights about the brain. I propose a framework in this dissertation that leverages the flexibility of deep generative models, which are data-driven unsupervised neural networks, and imbues them with more complex neurophysiological priors through inductive biases. The main two axes of neurophysiological priors I explore in this work are the incorporation of dynamics and multimodal information into deep generative models for neuroimaging. My goal is to show that by incorporating neurophysiological priors through inductive biases in deep generative models, the resulting low-dimensional representations are neurophysiologically informed and can be used to make inferences about dynamical or multimodal neuroimaging features. Throughout this work I also focus on the development of interpretability techniques that allow the structure of the low-dimensional representations and the features they capture to be visualized and used to understand what particular spatiotemporal patterns are implicated in cognitive processes or clinical populations. The framework I propose is evaluated on various task-based, resting-state, naturalistic, and clinical fMRI datasets in order to show the validity of both the neurophysiological priors and the framework as a whole. In many cases, I show that the framework is able to uncover either dynamically-informed or multimodally-informed brain patterns that are related to either behavioral variables and/or schizophrenia. Hence, I show that the proposed framework can be used to study cognitive processes, and has the potential to support biomarker development for a wider range of neuropsychiatric disorders.