Accurately mapping and evaluating hydrocarbon resources in the subsurface is critical to meeting the energy needs of our modern world. Hydrocarbons, such as oil and natural gas, serve as a major source of energy, a vital feedstock for various industrial processes, and play a significant role in global economics and geopolitics. Successful hydrocarbon exploration efforts depend on the integration of various scientific disciplines, including but not limited to geology, geophysics, geochemistry, petrophysics, and environmental geoscience. To accurately map and characterize subsurface deposits, exploration companies acquire and interpret various kinds of data, such as seismic shots, well logs, and core samples. These data have different natures, resolutions, scales, and extents. Additionally, they yield insights into overlapping, but also complementary characteristics of the subsurface. Over the recent years, deep learning has shown promise in automating subsurface understanding for exploration. However, a more complete adoption of advanced deep learning algorithms requires overcoming challenges on two fronts: firstly, acquiring training labels for machine learning models is an expensive and time-consuming process. Consequently, models trained on limited labeled data are prone to overfitting. Secondly, the efficient integration of various subsurface data depends on machine learning models that can account for their unique spatio-temporal properties. Our work has served to address these challenges in a systematic fashion by leveraging advanced machine learning paradigms such as active learning, by developing novel training frameworks suitable for settings involving limited, incomplete labels, and by building machine learning models to better account for the spatio-temporal properties of subsurface data.