Robotic systems must perceive, infer, and act under partial observability, nonlinear dynamics, and strict data and computational constraints. High-dimensional sensory inputs are often incomplete, heterogeneous, and asynchronous and require robust task control policies and adapting the underlying visual representations to the task. This dissertation investigates how to construct efficient, task-driven spatiotemporal representations that support reliable dynamics learning and policy execution from limited or suboptimal observations. Across four contributions, the work demonstrates that structured inductive biases grounded in dynamics, temporal modeling, and causal relevance significantly outperform generic multimodal encoders in realistic robotic settings. STEMFold learns a stochastic temporal manifold for multi-agent prediction with hidden agents, providing strong performance under extreme partial observability. RoboKoop integrates contrastive visual learning with Koopman operator theory to yield linear latent dynamics suitable for stable long-horizon control. MAPLE introduces a multimodal Mamba-based state-space model that fuses asynchronous event streams and RGB frames for robust visual reinforcement learning under challenging illumination and motion. AdaCred develops a causal credit-assignment mechanism that identifies task-relevant trajectory components, improving data efficiency in offline and demonstration-driven policy learning.Together, these frameworks show that compact, dynamics-aligned, and task-aware representations enable robust prediction and control in robotics while meeting real-world constraints on sensing, data, and computation.