In many high-stakes domains, the same observation may be associated with multiple plausible targets due to intrinsic data ambiguity and subjectivity of interpretation processes. The difference of ambiguous targets, referred to as disagreement, is conventionally considered as noise that needs to be averaged. This thesis argues that disagreement should not be simply suppressed. The distribution of disagreeing labels conveys scientifically meaningful data uncertainty and is also practically useful for downstream tasks. Thus, we view disagreement as an informative signal. We consider a unified generative perspective for modeling disagreement as estimating the distribution over all plausible labels conditioned on an input. The central challenges are representational and supervisory. Conditional variational models could suffer from collapsing distinct and informative label variation into a subset of latent directions in the representation space, degrading the diversity and structural fidelity of model-generated outputs. The supervisory challenge involves the exploitation of labels of heterogeneous quality. Models need to learn from heterogeneous-quality supervision without degrading reliability. We address the challenges through conducting representation analyses and introducing regularization mechanisms that encourage sufficient distinction between label-conditioned representations, while strengthening their alignment with informative target variation. We further demonstrate that disagreement-aware learning can be leveraged to estimate data ambiguity and augment scarce high-quality annotations in interpretation workflows. Finally, the same generative principle is extended to a non-deterministic sequence forecasting setting, where multiple plausible futures constitute a temporal analogue of target disagreement. Together, the thesis establishes disagreement as a meaningful distributional property of data and shows that explicitly modeling it can yield more faithful and practically useful learning systems.