The objective of the proposed research is to bridge the gap between 2D perception and reliable 3D scene understanding for real-world applications. While 2D images provide rich semantic information, their lack of explicit geometric structure limits their ability to support accurate spatial reasoning. To address this challenge, this research investigates methods for lifting 2D information into 3D representations through two complementary directions. First, the proposed work leverages high-quality 2D detection and segmentation, integrating them with spatial cues such as camera poses to infer consistent 3D structures across views. Building on prior work in transportation asset detection and extending to generative 3D reconstruction methods, this direction explores scalable, data-driven approaches for scene understanding from RGB imagery. Second, the research incorporates specialized sensing modalities, including event cameras, depth cameras, and laser-based sensors, which directly capture geometric information. These modalities enhance robustness and accuracy, particularly in challenging conditions where vision-only approaches may fail. The focus will be on leveraging depth for object reconstruction and laser sensing pavement distress detection. By integrating visual and geometric information across modalities, the proposed research aims to advance both the accuracy and scalability of scene understanding systems. The outcomes are expected to support critical applications in smart city infrastructure, including roadway condition assessment, asset monitoring, and large-scale environment modeling.